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Research  On  Automatic  Verification 
Of  Finite-State  Concurrent  Systems 


E.  M.  Clarke  and  O.  Griimberg 
Carnegie  Mellon  University,  Pittsburgh 

1 .  Introduction 

Temporal  logics  were  first  developed  by  philosophers  for  reasoning  about  the  ordering  of  events  in 
time  without  introducing  time  explicitly  [Hughes  &.  Creswel!  77).  Although  a  number  of  different 
temporal  logics  have  been  studied,  most  have  an  operator  like  G(f)  that  is  true  in  the  present  if /is 
always  true  in  the  future  (i<?„  if  /is  globally  true).  To  assert  that  two  events  ex  and  e2  never  occur  at  the 
same  time,  one  would  write  G(~ielv~<e1).  Temporal  logics  are  often  classified  according  to  whether 
time  is  assumed  to  have  a  linear  or  a  branching  structure.  This  classification  may  occasionally  be 
misleading  since  some  temporal  logics  combine  both  linear-time  and  branching-time  operators. 
Instead,  we  will  adopt  the  approach  used  in  [Emerson  &  Halpcrn  83]  that  permits  both  types  oflogics  to 
be  treated  within  a  single  semantical  framework.  In  this  papefthe  meaning  of  a  temporal  logic  formula 
will  always  be  determined  with  respect  to  a  labelled  state  transition  graph;  for  historical  reasons  such 
structures  arc  called  Kripke  models  [Hughes  &  Creswell  77]. 

Pnueli  was  apparently  the  first  person  to  use  temporal  logic  for  specifying  and  verifying  concurrent 
programs  [Pnculi  77].  His  approach  involved  proving  desired  properties  of  the  program  under 
consideration  from  a  set  of  program  axioms  that  described  the  behavior  of  the  individual  statements  in 
the  program.  Proofs  were  usually  constructed  by  hand,  and  this  task  was  in  general  quite  tedious.  Since 
many  concurrent  programs  can  be  viewed  as  communicating  finite  state  machines,  there  was  a  strong 
possibility  that  at  least  some  of  these  programs  could  be  automatically  verified.  I  he  first  verification 
technique  to  exploit  this  observation  was  the  CTL  model  checking  procedure  developed  by  Clarke  and 
Emerson  in  [Clarke  &  Emerson  81].  Their  algorithm  was  polynomial  in  both  the  sir.c  of  the  model 
determined  by  the  program  under  consideration  and  in  the  length  of  its  specification  in  temporal  logic. 
They  also  showed  how  fairness  (Gabbay  ct  al  80]  could  be  handled  without  changing  the  complexity  of 
•their  algorithm.  Handling  fairness  was  an  important  step  since  the  correctness  of  many  concurrent 
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algorithms  depends  critically  on  some  assumption  of  this  type;  for  example,  absence  of  starvation  in  a 
mutual  exclusion  algorithm  may  depend  on  the  assumption  that  each  process  makes  progress  infinitely 
often. 


At  roughly  the  same  time  Quiellc  and  Sifakis  [Quiclle  &  Sifakis  81]  gave  a  model  checking  algorithm 
for  a  similar  branching-time  logic,  but  they  did  not  analyze  its  complexity  or  show  how  t»  handle  an 
interesting  notion  of  fairness.  Later  Clarke,  Emerson,  and  Sistla  [Clarke  et  al  86a]  devised  an  improved 
algorithm  that  was  linear  in  the  product  of  the  length  of  the  formula  and  in  the  size  of  the  global  state 
graph.  Sistla  and  Clarke  [Sistla  &  Clarke  86]  analyzed  the  model  checking  problem  for  a  variety  of  other 
temporal  logics  and  showed,  in  particular,  that  for  linear  temporal  logic  the  problem  was  PSPACE 
complete. 

A  number  of  papers  have  shown  how  the  temporal  logic  model  checking  procedure  can  be  used  for 
verifying  network  protocols  and  sequential  circuits  ( [Clarke  et  al  86a],  [Mishra  &  Clarke  85],  [Browne  et 
al  86],  [Dill  &  Clarke  86],  [Browne  et  al  85],  [Browne  &  Clarke  86],  [Browne  et  al  6b]).  In  the  case  of 
sequential  circuits  two  approaches  have  been  developed  for  obtaining  state  transition  graphs  to  analyze. 
The  first  approach  extracts  a  state  graph  directly  from  the  circuit  under  an  appropriate  timing  model  of 
circuit  behavior.  The  second  approach  obtains  a  state  transition  graph  by  compilation  from  a  high  level 
representation  of  the  circuit  in  a  Pascal-like  programming  language.  In  practice  the  model  checking 
procedure  is  able  to  check  state  transition  graphs  at  a  rate  of  100  states  per  second  for  formulas  of 
reasonable  length.  It  has  been  used  successfully  to  find  previously  unknown  errors  in  published  designs 
of  asynchronous  circuits. 

Alternative  approaches  have  been  proposed  by  a  number  of  other  researchers.  Ihe  approach  used  by 
Kurshan  [Kurshan  86]  involves  checking  inclusion  between  two  automata  on  infinite  tapes.  The  first 
machine  represents  the  system  that  is  being  verified;  the  second  represents  its  specification.  Automata 
on  infinite  tapes  are  used  in  order  to  handle  fairness.  Pnueli  and  Lichtenstein  [Lichtenstein  &  Pnueli 
85]  reanalyzed  the  complexity  of  checking  linear-time  formulas  and  discovered  that  although  the 
complexity  appears  exponential  in  the  length  of  the  formula,  it  is  linear  in  the  u/c  of  the  global  state 
graph.  Based  on  this  observation,  they  argued  that  the  high  complexity  of  linear-time  model  checking 
might  still  be  acceptable  for  short  formulas.  Emerson  and  Lei  [Emerson  &  Lei  85]  extended  their  result 
to  show  that  formulas  of  the  logic  CTL*,  which  combines  both  branching-time  and  linear-time 
operators,  could  be  checked  with  essentially  the  same  complexity  as  formulas  of  linear  temporal  logic. 
Vnrdi  and  Wolpcr  have  recently  [Vardi  Sc  Wolper  86]  shown  how  the  model  checking  problem  can  be 
formulated  in  terms  of  automata,  thus  relating  the  model  checking  .ippm.iji  to  the  woi  k  of  Kurshan. 
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Although  the  model  checking  procedure  discussed  in  this  paper  has  already  been  used  to  discover 
some  surprising  errors  in  non-trivial  programs,  mote  work  still  remains  to  be  done.  Certainly  the  most 
serious  problem  is  the  stale  explosion  problem.  In  analyzing  a  system  of  ,V  processes,  the  number  of 
states  in  tire  global  state  graph  may  grow  exponentially  with  .V.  Recent  research  indicates,  however, 
that  it  may  be  possible  to  avoid  this  problem  in  some  important  cases.  For  instance,  techniques 
developed  in  [Clarke  ct  at  86b]  may  reduce  the  size  of  the  state  graph  that  needs  to  be  searched  when 
many  of  the  processes  arc  identical.  It  may  also  be  possible  to  exploit  the  hierarchical  structure  of  a 
complex  concurrent  program  in  order  to  reduce  die  number  of  states  diat  need  to  be  considered  at  any 
one  level  of  abstraction  [Mishra  &  Clarke  85J. 

This  survey  is  organized  as  follows:  Section  2  describes  the  syntax  and  semantics  of  the  temporal 
logics  that  are  used  in  this  paper.  In  Section  3  we  state  the  model  checking  problem  and  give  an  efficient 
algorithm  for  checking  simple  branching-time  formulas.  In  Section  4  we  discuss  tire  issue  of  fairness 
and  show  how  the  algoridirn  of  Section  3  can  be  extended  to  include  fairness  constraints.  Section  5 
demonstrates  how  the  model  checking  algorithm  can  be  used  to  debug  a  simple  mutual  exclusion 
program.  In  Section  6  we  describe  some  alternative  approaches  for  verifying  systems  of  finite  state 
concurrent  processes.  We  analyze  the  complexity  of  checking  linear  temporal  logic  formulas  and  outline 
die  techniques  of  Pnucli  and  Lichtenstein  [Lichtenstein  &  hwcli  85]  and  Vardi  and  Wolper  [Yardi  & 
Wolper  86], Additional  applications  to  circuit  and  protocol  verification  arc  discussed  in  Section  7.  The 
paper  concludes  in  Section  8  with  a  discussion  of  some  of  the  important  remaining  research  problems 
like  the  stale  explosion  problem. 

2.  Computation  Tree  Logics 

In  this  paper  finite  state  programs  are  modelled  by  labelled  state-transition  graphs,  called  Kripke 
strvcturcs  [Hughes  &  Crcswcll  77],  If  some  suite  is  designated  as  die  initial  state,  then  the  Kripke 
structure  can  be  unwound  into  an  infinite  tree  with  that  suite  as  the  root.  Since  paths  m  the  tree 
represent  possible  computations  of  the  program,  we  will  refer  to  die  infinite  tree  obtained  in  this 
manner  as  the  computation  tree  of  the  program.  Temporal  logics  may  differ  according  to  how  they 
handle  branching  in  the  underlying  computation  tree.  In  linear  temporal  logic,  operators  arc  piowded 

for  describing  events  along  a  single  computation  path.  In  a  brandling-time  logic  die  temporal  operators 

* 

quantify  over  the  paths  that  arc  possible  from  a  given  state.  The  computation  tree  W.u  ('ll 
(  [l-'incrson  &  Clarke  81],  [F.mcrson  &  Halpcrn  81],  [Clarke  et  a!  86a])  combines  both  branching-time 
and  linear-time  operators;  a  path  quantifier,  either  A  ("for  all  computation  p  stlts")  or  I’  ("f-r  m  me 
computation  path")  can  prefix  an  assertion  u imposed  of  .nbitr.iiv  combinations  o:  the  usual  line  a  time 
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operators  G  ("always").  F  ("sometimes").  X  ("nexttime").  and  U  ("until").  The  remainder  of  tins 
section  gives  a  precise  description  of  the  syntax  and  semantics  of  these  logics. 

There  are  two  types  of  formulas  in  CTL  :  slate  formulas  (which  are  true  in  a  specific  state)  and  path 
formulas  ( which  arc  true  along  a  specific  path).  Let  IP  be  the  set  of  atomic  proposition  names.  A*statc 
formula  is  either: 

•  A, if  AtAP. 

•  If  /  and  g  are  state  formulas,  then  -> /  and  /  v  g  are  state  formulas. 

•  If  /  is  a  path  formula,  then  K(/)  is  a  state  formula. 

A  path  formula  is  either: 

•  A  state  formula. 

•  If  /  and  g  are  path  formulas,  then  ->/  /  v  g ,  X/  and  f\J  g  are  path  formulas. 

CTL'  is  the  set  of  state  formulas  generated  by  the  above  rules. 

CTL  ( [Ben-An  et  al  83],  [Clarke  &  Emerson  81])  is  a  restricted  subset  of  CTL*  that  permits  only 
branching-time  operators-- each  path  quantifier  must  be  immediately  followed  by  exactly  one  of  the 
operators  G,  F,  X,  or  U.  More  precisely,  CTL  is  the  subset  of  CTL*  that  is  obtained  if  the  path  formulas 
arc  restricted  as  follows: 

•  If  /  and  g  are  state  formulas,  then  X  /and  /U  g  are  path  formulas. 

•  If  /  is  a  path  formula,  then  so  is  -»/ 

Linear  temporal  logic  (LTL),  on  the  other  hand,  will  consist  of  formulas  that  have  the  form  A /  where 
/  is  a  path  formula  in  which  the  only  state  subformulas  that  are  permitted  are  atomic  propositions. 
More  formally,  a  path  formula  is  either 

•  An  atomic  proposition 

•  If  /  and  g  are  path  formulas,  then  ->/  /  v  g ,  X/  and  f\J  g  arc  path  formulas. 

We  define  the  semantics  of  CTL*  with  respect  to  a  structure  M = <S,  R,  L> ,  where 

•  S'  is  a  set  of  states. 

•  flCSxS  is  the  transition  relation,  which  must  be  total.  We  write  r,  — >  s,  to  indicate  that 
(Vi)**- 

•  L :  S—*9(AP)  is  a  function  that  labels  each  state  with  a  set  of  atomic  propositions  true  in 


that  state. 


Unless  otherwise  stated,  all  of  our  results  apply  only  to  finite  Kripkc  structures. 

We  define  a  path  in  M  to  be  a  sequence  of  states,  77  -  $,.s, _ such  that  for  every  /  >  0.  5,-*  77' 

will  denote  tlie  suffix  of  77  starting  at  sr 

We  use  the  standard  notation  to  indicate  that  a  state  formula  f  holds  in  a  structure:  3/.j)=  /means 
that  /  holds  at  suite  s  in  structure  .1/.  Similarly,  if  /  is  a  path  formula.  M.n  t=f  means  that  f  holds 
along  path  u  in  structure  M.  I  lie  relation  is  defined  inductively  as  follows  (assuming  that  /  and/ 
arc  state  formulas  and  g ,  and  g  2  are  path  formulas): 

1.  sN=.l  <=>  Ail.(s). 

2.  sh=->/  «=>  sM5/. 

3.  st=/v/  «  sN/orsN/. 

4.  st=  K(g,)  «=»  there  exists  a  path  v  starting  with  ssuch  that  w  1=  gv 

5.7 rh=/  «=>  s  is  the  first  state  of  7r  and  sN/. 

6.  it  !=  -<gt  <=>  v  ht  gr 

7-  tt  gt  v  g2  «=>  77 1=  g!  or  77  N  gr 

8.  it  Xgj  <=»  n 1  k®8  gj. 

9. 77 1=  g1  U  g2  <=>  there  exists  a  £  >0  such  that  Trk  \=  g.  and  for  all  0  <j  <  k,  i r'h=g[. 

Wc  will  also  use  the  following  abbreviations  in  writing  CTL*  (CTL  and  !  'l  l .)  formulas: 

•/A  g  =  ->(-’/ v  ~<g)  •  F/  =  true  U/ 

•  A(/)  =  -K(V)  •  G/  ==  — 1 F— '/. 

In  ( [Lamport  80],  [F.mcrson  &  Haipcrn  83[)  it  is  shown  that  the  three  logics  discussed  in  this  section 
have  different  expressive  powers.  Lor  example,  there  is  no  CTI.  formula  that  is  equivalent  to  the  LTL 
formula  ,1(FGp).  Likewise,  there  is  no  1.11.  formula  that  is  equivalent  to  the  CTL  formula 
AG(  FFp).  The  disjunction  of  these  two  formulas  l(  FGp)v  AG(  IT>)  is  a  C  LL*  formula  that  is  not 
expressible  in  either  CTL  or  LTL. 

3.  The  CTL  Model  Checking  Algorithm 

1  ct  M  ~  (.V.  /?,  /.)  be  a  finite  Kripkc  stnictmc.  Assume  ih.it  we  want  to  determine  which  suites  m  A 
satisfy  tlie  Cl  1.  formula  /.  We  will  design  mir  algorithm  to  operate  in  slams;  I  he  first  si.i.-c  paxesses 
all  subformulas  of  /  oflcneth  1.  the  second  stage  processes  ill  suMonimi.i  • .  T  Vncth  ?.  ;r,l  •<>  ->n  \( 
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the  end  of  the  ilh  stage,  each  state  will  be  labeled  with  the  set  of  all  subfoi  mulas  of  length  less  than  or 
equal  to  i  that  are  taic  in  the  state.  We  let  the  expression  Iab<’!(s)  denote  this  set  for  state  s.  When  the 
algorithm  terminates  at  the  end  of  stage  n  =  lengih(f^) ,  we  see  that  for  all  suites  and  for  all 
sc.bformulas/of/,  M.s  t=  /iff  /e  label(s)  . 

Observe  that  AX  can  be  expressed  in  terms  of  KX  and  that  ALi  can  be  expressed  in  terms  of  KL  and 
F.G: 

AX/  ==  -'KX  ~'fl 

At/;  U/j]  =  -'(K(->/ 0  (-1/  A  ->/))  v  KG( ->/)). 

1  hus,  for  the  stage  i  algorithm  it  is  sufficient  to  be  able  to  handle  six  eases,  depending  on  w  bother  /  is 
atomic  or  has  one  of  the  following  forms:  ->/, /v/,  KX/,  K[/lj/]  or  KG/. 

We  will  only  consider  die  last  two  cases,  since  the  others  arc  straightforward. 

To  handle  formulas  of  the  form  /=  K[/  U /]  we  first  find  all  of  those  suites  that  arc  labeled  with  / . 
We  then  work  backwards  using  d'e  converse  of  die  transition  relation  R  and  find  all  of  diosc  states  that 
can  be  reached  by  a  path  in  which  each  state  is  labeled  with  /.  All  such  states  should  be  labeled  with  /. 
This  step  requires  time  0(|  ,S|  + 1  R  | ). 

The  ease  in  which  /=  FG /  is  slightly  more  complicated  and  depends  on  die  following  observation. 

I.emma  l:  Let  SP  be  obtained  from  A/  by  deleting  from  5  all  of  diose  states  at  which  /  does  not 
hold  and  restricting  R  and  I.  accordingly,  thus,  M'  -  (S',  R',  L')  where  S'  =  {sc  S\M, v  1=  /  }•, 
R'  -  /?|  s'  xs'  •  and  l.'  -  /.I5' •  Then  M.s  1=  KG/  iff  the  following  two  conditions  are  satisfied: 

1.  s'.  S' 

2.  there  exists  a  path  in  S'  that  leads  from  s  to  some  node  1  in  a  non-irnu;!  stiongly  connected 
componenr  of  die  graph  (S'  R'). 

Proof:  Assume  that  Ms  M=  KG/.  Clearly  siS'  .  1  et  77  be  an  infinite  path  suiting  at  s  such  that  / 
holds  at  each  state  on  v  .  Since  \l  is  finite,  it  must  be  possible  to  write  v  as  v  —  77  ,v  ■  where  tt  .  is  a 
finite  initial  segment  and  w,  is  an  m finite  suffix  of  n  with  die  property  that  cadi  slate  on  rt  occurs 
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infinitely  often.  Obviously  77  0  is  contained  in  S' .  Let  C  be  the  set  of  states  in  vx.  C  is  a  nontrivial 
strongly  connected  component  of  S' .  To  see  this,  let  jj  and  s2  be  states  in  C.  Pick  some  instance  of  s2 
on  7T, .  By  the  way  in  which  was  selected,  we  know  that  there  is  an  instance  of  52  further  along  v. . 
The  segment  from  to  s2  lies  entirely  within  C  and  hence  within  S' .  This  segment  is  a  finite  path 
from  5,  to  .s,  in  S' .  Thus,  both  condition  (1)  and  condition  (2)  arc  satisfied. 

Next,  assume  that  conditions  (1)  and  (2)  are  satisfied.  Let  tt.  be  the  path  from  s  to  /.  l  et  w2  be  a 
finite  path  of  length  at  least  one  that  leads  from  /  back  to  /.  1  he  existence  of  w,  is  guaranteed  since  C  is 
a  non-trivial  strongly  connected  component.  All  of  the  states  on  the  infinite  path  it  =  n  tt.u  satisfy  f . 
Since  it  is  also  a  possible  path  starting  at  s  in  M.  we  sec  that  \t,i r  1=  F.G  f .  □ 

The  algorithm  for  the  case  of  /  =  Y.Gf  follows  directly  from  the  lemma.  We  construct  the  restricted 
Kripke  structure  M'  =  (S',  R' ,  L')  as  described  in  the  statement  of  the  lemma.  We  partition  the 
graph  (S' ,  R')  into  strongly  connected  components  and  find  those  states  that  belong  to  nontrivial 
components.  We  then  work  backwards  using  the  converse  of  R  and  find  all  of  those  states  that  can  be 
reached  by  a  path  in  which  each  state  is  labeled  with  f .  This  step  also  requires  time  0(  |  .S’|  +  |  R  \ ). 

In  order  to  handle  an  arbitrary  CTL  formula  l ",  wc  successively  apply  the  state  labeling  algorithm  to 
die  subformulas  of  f0,  starting  with  the  shortest,  most  deeply  nested  and  work  outward  to  include  all  of 
J, ].  Since  each  pass  takes  time  0(|  f>'|  +  |  R\ )  and  since  f0  has  len^ih(f0)  different  subl'ormulas,  die 
entire  algoridim  requires  O(length(f0)-  ( |  S'|  -F  |  R  | )). 

Theorem  2:  There  is  an  algorithm  for  determining  whether  a  CTL  formula  f0  is  true  in  state  s  of  the 
structure  M  =  (S,  R,  L)  that  runs  in  time  0(len%ih(f0)-  (|  S|  +  |  R  \ )). 

4.  Fairness  Constraints 

In  verifying  concurrent  systems,  we  are  occasionally  interested  only  in  correctness  along  fair 
execution  sequences.  For  example,  with  a  system  of  concurrent  processes  we  may  wish  to  consider  only 
those  computation  sequences  in  which  each  process  is  executed  infinitely  often.  When  dealing  with 
network  protocols  where  processes  communicate  over  an  imperfect  (or  lossy)  channel  wc  may  also  wish 
to  restrict  the  set  of  computation  sequences;  in  this  case  die  unfair  execution  sequences  are  those  in 
which  a  sender  process  continuously  transmits  messages  without  any  reaching  die  receiver  due  to  erratic 
behavior  by  die  channel. 

It  on  'hlv  speak  inn  a  fairness  condition  |SSci  is  that  requests  lor  sen  ice  ate  •”  mted  ”  ml  Ik  u'ui!\  <  a  u" 
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Different  concepts  of  what  constitutes  a  "request"  and  what  "sufficiently  often"  should  mean  gi\ c  rise 
to  a  variety  of  notions  of  fairness.  Indeed,  many  different  types  of  fairness  and  approaches  to  dealing 
with  them  have  been  proposed  in  the  literature:  we  refer  the  reader  to  [Gabbay  ct  al  SO],  [I  amport  80). 
[Quiclle  &  Sifakis  82],  and  (I  ehrnann  ct  al  81)  for  more  extensive  treatments.  I'he  text  by  France/ 
[France/  86)  also  gives  an  excellent  survey  of  the  various  tv  pes  of  fairness. 

In  diis  section  we  will  show  how  to  extend  the  Cl  L  model  checking  algorithm  to  handle  a  simple  but 
fundamental  type  of  fairness  in  which  ceitain  predicates  must  hold  infinitely  often  along  every  lair  path. 
(  [Clarke  et  al  86a|  shows  how  to  handle  a  richer  class  of  fairness  constraints.)  In  this  case  it  follows  from 
[F.tncrson  &  Halpcrn  S3)  that  correctness  of  fair  executions  cannot  be  expressed  in  C 11  . 

In  order  to  handle  fairness  and  stall  obtain  an  efficient  model  checking  algorithm  we  modify  the 
semantics  of  CTL.  I'he  new  logic,  which  we  call  CI  l/  ,  has  the  same  syntax  as  Cl  I..  But  a  structure  is 
now  a  4-tuple  .V/  =  (S.  R,  L,  F)  where  .S'.  R,  /.  have  the  same  meaning  as  in  the  case  of  CTL,  and  Fis  a 
collection  of  predicates  on  5,  FC  2s.  A  path  v  is  F-Jhir  iff  die  follow  ing  condition  holds:  for  each 
G  €  F,  i here  are  infinitely  many  states  on  v  which  satisfy  predicate  G.  Cl  l/  has  exactly  tire  same 
semantics  as  CTL  except  that  all  path  quantifieis  range  over  fair  paths.  The  first  step  in  checking  C  I'l/ 
formulas  is  to  determine  the  fair  strongly  count  cted  components  of  the  graph  of  \f.  A  strongly  connected 

component  is  fair  if  it  contains  at  least  one  state  from  each  set  in  F.  Formally,  let  F  =  { G . <iK)  be  a 

collection  of  subsets  of  .S'.  A  strongly  connected  component  C  of  the  graph  of  M  is  fair  iff  for  each  O',  in 
F ,  there  is  a  state  /,  e  (Cn  (7,). 

Lemma  3:  Given  any  finite  structure  M  =  (S.  R.  L.  F)  where  F  is  a  set  of  fairness  constraints  and  a 
state  %  e  S',  the  following  two  conditions  arc  equivalent: 

1.  There  exists  an  F-ydir  path  in  M  starting  at 

2.  There  exists  a  fair  strongly  connected  component  (  of  (the  graph  of)  \l  such  that  theie  is  a 
finite  path  from  Sg  to  a  state  t  e  C. 

The  proof  is  straightforward  and  is  given  in  [Clarke  et  al  86a).  We  next  extend  our  model  checking 

C 

algorithm  to  CTL  .  We  introduce  an  additional  proposition  (J.  which  is  true  at  a  state  iff  there  is  a  fair 
path  starting  from  that  state.  This  can  easily  he  done.  by  obtaining  the  stiongly  connected  components 
of  the  graph  associated  with  the  uructure  and  marking  a  component  ..s/m;  ;f  it  contains  at  least  one 
state  from  each  G,  in  /'.  By  the  above  lemma  every  state  m  a  fair  strong!,  connected  component  is  the 
start  of  an  infinite  fair  padi.  Thus,  we  label  a  stole  wuii  <J  of  theie  is  a  path  from  drat  state  to  M>me 


( 
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node  of  a  fair  strongly  connected  component.  As  usual  wc  design  the  algorithm  so  that  after  it 
terminates  each  state  will  be  labeled  with  the  subformulas  of  fQ  true  in  that  state.  We  consider  the  two 
interesting  cases  where  /  is  a  subformula  of  /  and  cither  /  =  K[/U/]  or  f  -  F.G/.  Wc  assume 
that  the  states  have  already  been  labeled  with  the  immediate  subformulas  of  /by  an  earlier  stage  of  the 
algorithm. 

1  ./=  Y[f  U/  ] :  /is  true  in  a  state  iff  the  CTI.  formula  F[/l;(/jA@)|  is  true  in  that  state, 
and  this  can  be  determined  using  the  Cl'I.  model  checker.  Again,  state  s  is  labeled  with  /iff 
/is  true  in  that  state. 

2.  /-  FG( /,):  To  determine  if  sN  KG(/j)  we  use  tire  procedure  described  in  section  3  to 
check  FG ( /  A  Q)  in  the  structure  with  the  additional  proposition  Q. 

It  is  easy  to  see  that  the  above  algorithm  runs  in  time  0(!ength  (f0 )  ( 1 6'  |  +  |  R  \ )) . 

Theorem  4:  There  is  an  algorithm  for  determining  whether  a  CTL  formula  /  is  true  in  state  s  of  the 
structure  A/  =  (5,  R,  L,  F)  with  F  as  the  set  of  fairness  constraints  that  ains  in  time 
(Klength(f0)-(\S\  +  |*l)). 

5.  An  Example 

In  this  section  wc  illustrate  how  the  model  checker  can  be  used  to  verify  a  simple,  but  not  entirely 
trivial,  concurrent  program.  The  example  is  a  two  process  mutual  exclusion  program  that  was  manually 
proved  correct  using  linear  temporal  logic  by  Owicki  and  Lamport  in  [Owicki  &  Lamport  82].  Flic 
program,  expressed  in  a  variant  of  the  CSP  programming  language  [Hoarc  73],  is  shown  in  Figure  5-1. 
In  this  version  of  CSP  processes  may  have  global  variables  (e.g.  pi  and  p2),  and  assignments  to  such 
variables  arc  assumed  to  be  atomic.  Since  our  verification  technique  can  only  be  used  to  analyze  finite 
state  concurrent  systems,  we  require  that  all  variables  be  boolean  and  that  all  messages  between 
processes  be  signals.  Labels  (e.g.  NCI  and  NC2)  are  used  to  indicate  that  flow  of  control  has  reached  a 
particular  point  in  some  process.  In  our  example  there  arc  two  processes  SI  and  S 2.  and  each  process 
has  three  code  regions:  a  nonchtical  region  NCi  in  which  the  process  computes  some  data  values  that  it 
wishes  to  share  with  the  other  process,  a  trying  region  7 7  in  which  the  process  executes  a  protocol  to 
obtain  entry  into  the  critical  section,  and  a  critical  section  CSi  in  which  the  pioccss  updates  shared 
variables.  To  prevent  a  race  condition  that  might  result  in  unpredictable  values  being  assigned  to  (he 
shared  variables,  only  one  process  is  allowed  to  be  in  its  critical  section  at  any  given  time.  Note  th.it  tire 
two  processes  arc  different;  hence  this  is  not  a  symmetric  solution  to  the  mutual  exclusion  problem. 
When  the  CSP  program  is  compiled  a  slate  graph  with  77  states  is  obtained.  Milnuigh  this  not  an 
extremely  large  state  machine,  it  would  neveillieless  be  qua*,  tedious  tor  a  lium.ui  to  debug. 


We  initially  am  the  verifier  without  any  fairness  constraints--See  Figure  5-2.  Wc  first  check  to  see  if 
both  processes  arc  ever  in  their  critical  regions  at  the  same  time.  This  property  is  succintly  expressed  by 
the  CTI.  formula  FF(CA1  A  CS2).  The  verifier  rapidly  determines  that  the  formula  is  falsc-hcncc,  the 
program  does  guarcntcc  mutual  exclusion.  Time  is  measured  in  1/60  of  a  second.  The  first  component 
measures  userepu  time.  The  second  component  measures  system  epu  time.  Wc  next  check  for  absence 
of  deadlock.  This  is  expressed  by  the  formula  AG(FF(C.S1  v  CS2)).  The  verifier  determines  that  this 
formula  is  satisfied;  thus,  from  any  state  that  is  reachable  from  the  initial  state  it  is  always  possible  to  get 
to  cither  CA  /  or  CS2. 


Absence  of  starvation  for  process  1  is  expressed  by  the  formula  AG  (71  — >  AF  CAT) .  This  property  is 
not  satisfied  without  a  fairness  constraint.  The  reason  is  quite  simple.  When  wc  build  the  global  state 
graph  for  the  program  w'e  do  not  make  any  assumptions  about  the  relative  speeds  of  the  two  processes. 
Thus,  die  second  process  can  make  any  number  of  steps  between  steps  of  die  first  process.  In  fact,  the 
second  process  can  even  run  forever,  thereby  preventing  die  first  process  from  ever  making  another 
step.  Wc  can  rule  out  the  second  type  of  behavior  by  means  ot' fairness  constraints  which  require  diat 
each  process  be  given  a  chance  to  execute  infinitely  often.  In  Figure  5-3  we  restart  the  verifier  with 
several  fairness  constraints  that  prevent  cither  process  from  remaining  forever  at  die  same  statement 
while  enabled  to  make  a  step.  Under  these  assumptions  the  first  process  will  never  starve.  I  lowcvcr.  the 
possibility  of  starvation  sdll  exists  for  the  second  process. 

A  good  solution  to  the  mutual  exclusion  problem  should  not  require  that  processes  alternate  entry 

into  dicir  critical  regions;  CA1,  CS2,  CAT,  CA'2 . In  order  to  test  that  the  algorithm  given  in  Figure 

5-1  does  not  require  strict  alternation,  wc  check  die  formula 
AG  (CAT  -  A[CA1  U(->CA1  A  A[->CAT  U  CA2| )  ] ). 

This  formula  asserts  that  if  process  1  enters  its  critical  section  and  subsequently  leaves  it,  then  it  cannot 
enter  it  again  until  process  2  has  entered  its  critical  section.  The  verifier  determines  that  die  formula  is 
false  in  less  than  a  second.  This  example  shows  how  die  basic  temporal  operators,  particularly  the  until 
operator,  can  be  nested  to  express  complicated  timing  properties. 


Finally,  the  verifier  has  a  counterexample  feature  (that  is  not  shown  in  the  transcripts).  When  this 
feature  is  enabled  and  the  model  checker  determines  that  a  formula  is  false,  it  will  attempt  to  find  a  padi 
in  the  state  graph  which  demonstrates  that  the  negation  of  die  formula  is  true.  For  example.  if  the 
formula  has  the  form  AG(/),  our  system  will  produce  a  path  to  a  suite  in  which  -/  holds.  For 
instance,  when  die  verifier  determines  that  die  last  formula  above  is  false,  it  prints  out  an  execution  of 


pi , p2  :  bool  ; 

NCl,NC2,Tl,T2,T2a,CSl,CS2:  label ; 

[ 

SI  ,  S  2  :  process; 

SI  ::  [ 

pi  : =  false; 

*[ 


true  -> 

<<NC1>>  skip;  --noncri tical  section  1 
pi  : *  true; 

<<T1>>  *[  p2  ->  Skip]; 

<<CS1>>  skip;  --critical  section  1 
pi  : =  false 

] 


S2  ;  ;  [ 

p2  :=  false; 

•c 

true  -> 

<<NC2>>  skip;  - -noncr i t i cal  section2 
p2  :=  true; 

«T2>>  *[  pi  -> 

p2  : 5  false; 

<<T2a>>  *[pl  ->  skip  ]; 

p2  :-  true 

]: 

<<CS2>>  skip;  --critical  section  2 
p2  :=  false 

] 

] 

] 


] 


Figure  5*1:  Two  process  mutual  exclusion  program. 
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|=  EF(CS1  &.CS2). 

The  equation  is  FALSE. 


time:  (2  4) 


|=  AG(EF(CSl  |  CS2)). 
The  equation  is  1  RUE. 


time:  (4  2) 


|=  AGO  l  ■>  AFCSl). 
The  equation  is  FALSE. 


time:  (17  12) 


Figure  5-2:  Transcript  of  model  checker  execution  (without  fairness 

constraint). 


Fairness  constraint:  ~NC1. 

Fairness  constraint:  -NC2. 

Fairness  constraint:  ~CS1. 

Fairness  constraint:  -CS2. 

Fairness  constraint:  ~T1  |  P2. 
Fairness  constraint:  ~T2  |  pi. 
Fairness  constraint:  ~T2  |  ~pl  |T2a. 
Fairness  constraint: . 


|=  AG(T1  ->  AFCSl). 
1  he  equation  is  TRUE. 


time:  (10  0) 


|=  AG(T2->  AFCS2). 
The  equation  is  FALSE. 


time:  (29  9) 


|=  AG(  CSl->  A[CSl  U(~CS1&A(~CSI  UCS2])1). 


The  equation  is  FALSE. 


time:  (38  17) 


Figure  5-3:  Transcript  of  mode!  checker  execution  (with  fairness  constraint). 
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the  mutual  exclusion  program  in  which  process  1  enters  its  critical  region,  leaves,  and  reenters  without 
process  2  entering  its  critical  section  in  die  meantime.  This  feature  is  quite  useful  for  debugging 
purposes. 

6.  Other  Approaches 

Several  papers  have  considered  the  model  checking  problem  for  linear  temporal  logic  formulas.  l  et 
M  =  (.S',  R,  I.)  be  a  Kripke  Structure  with  50  €  S.  and  let  A/ be  a  linear  temporal  logic  formula.  '1  bus. 
/is  a  restricted  path  formula  in  which  the  only  state  subformulas  are  atomic  propositions.  We  wish  to 
determine  if  \f,s0  f=  A/.  Notice  that  M.s  N  A/iff  .If.s  M=  — •  — > /.  Consequently,  it  is  suiliuent  to  be 
able  to  check  die  truth  of  formulas  of  the  form  K/  where /is  a  restricted  path  formula.  In  general,  dm 
problem  is  PSPACK-complctc  [Sistla  &  Clarke  86],  Although  the  proof  of  this  PSPACT -completeness 
result  is  beyond  the  scope  of  our  survey,  it  is  easy  to  see  diat  the  model  checking  problem  is  NP-hard 
for  formulas  of  the  form  E/ where  /is  restricted  path  formula.  We  show  diat  the  directed  Hamiltonian 
path  problem  is  reducible  to  the  problem  of  determining  whether  \l.s^=f  where 

•  M  is  a  finite  structure, 

•  s  is  a  state  in  .1/  and 

•  /is  the  asseruon  (using  atomic  propositions  pl . pn) : 

K[Kp,A  ...  A  l'  pn  a  G(p,  —  XG->p.)A  •••  AG  (pn~*  XG^/>„)|. 

Consider  an  arbitrary  directed  graph  G  =  ( V.  A)  where  V  -  {v. . vj  .  We  obtain  a  structure 

from  (7  by  making  proposition  p,  hold  at  node  v;  and  false  at  all  other  nodes  (for  1  <  i <  n ),  and  by 
adding  a  source  node  u,  from  which  ail  v,  are  accessible  (but  not  vice  versa)  and  a  sink  node  u2  which 
is  accessible  from  all  v,  (but  not  vice  versa),  l-'oimally,  let  the  structure  \f  =  ( (J.  B.  I .)  consist  of 

U  =  where  uvuitV; 


B  =  /lU{(u;.  v,)|v;  e  l]U{(v#,  u:)\v,i  HU  {(to,  i/.)};  and 


L  is  an  assignment  of  propositions  to  suites  such  that 

•  p ,  is  true  in  v,  for  1  <i<n 

•  p,  is  false  in  v,  for  1  <  ij<  n,  t+ j 

•  pt  is  false  in  u  .u,  for  1  <  i<,  n 


3 


If 
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It  is  easy  to  sec  that  A l.ul  >=  /iff  there  is  a  directed  infinite  path  in  M  starting  at  u.  which  goes  through 
all  v,  €  V  exactly  once  and  ends  in  the  self  loop  though  w:.  Note  that  the  formula  /in  the  above 
construction  has  essentially  the  same  si/c  as  the  as  the  graph  G.  Suppose  that  the  length  of  the  formula 
to  be  checked  was  known  to  be  much  smaller  than  the  size  of  the  Kripke  structure  under  consideration. 
Would  the  complexity  still  be  high  in  this  ease?  A  careful  analysis  by  l.ichtcnstcin  and  l’nueli 
[I  ichtenstein  &  Pnucli  85]  showed  that  although  the  complexity  is  apparently  exponential  in  the  length 
of  the  formula,  it  is  linear  in  the  size  of  the  global  state  graph.  We  briefly  describe  their  results  below. 

I. et /be  a  restricted  path  formula,  the  cloture  off  (1(f).  is  tire  smallest  set  of  formulas  containing  f 
and  satisfying: 

•  -/  c  C/.(f)  iff  /  e  Cl.(J') 

•if/v/t  f/(/),thcn  /./er/(/) 

•  if  X/  e  CL(f).  then  /  e  Cl.(f) 

•  if  -X/  e  a.U  ).  then  X-/  e  Ci(f) 

•  if /U/  e  CL(f),  then  /./.  XtfU/l  e  CUJ) 

It  can  be  shown  that  the  size  of  Cl  (f)  is  5  •  length!  f). 

An  atom  is  a  pair  A  =  (sA,  FA)  with  j^eS  and  /•  ^CC^.(/)u<l/,  such  that: 

•  for  each  proposition  Q  e  Al\  Q  €  l'A  iff  Q  €  UsA) 

•  for  every  /  e  CL(f),  /  €  FA  iff  ->/  i  FA 

•  for  every  /,/  e  CL(f).  /v/e  FA  iff  /  or  /  €  FA 

•  for  every  -X/  €  CL(f),  -X/  €  FA  iff  X-/  e  F A 

•  for  every  /,/  e  CL(f),f\Jf  a  FA  iff  /  e  FA  or  /,  X[/;U/1  e  I  A 

Now,  a  graph  G  is  constructed  with  the  set  of  atoms  as  the  set  of  vertices.  (A.B)  is  an  edge  of  G  iff 
(5^.  sR)  €  R  and  for  every  formula  /,  if  X/e  /•/,  then  /  e  /•/.  An  eventuality  sequence  is  an  infinite 
path  1 r  in  G  such  that  if  /U/  e  FA  for  some  atom  zl  on  v  ,  then  there  cxims  an  atom  /?.  reachable 
from  -I  along  m  ,  such  that  /  €  If. 

lemma  5:  A/.v  b—  V  f  iff  there  exists  an  eventuality  sequence  starting  at  an  atom  (s,  F)  such  that 
ft  /'. 


A  non-trivial  strongly  connected  component  C  of  the  graph  G  is  said  to  be  self-fulfilling  iff  for  every 
atom  /!  in  C  and  for  every  f\Jf  e  FA  there  exists  an  atom  il  in  C  such  that  f  e  FB. 

Lemma  6:  M.s  N  F./  iff  tlicrc  exists  an  atom  A  =  (s.  /•)  in  G  such  that  ft  F  and  there  exists  a  path 
in  (7  from  .1  to  a  self-fulfilling  strongly  connected  component. 

l  emma  6  be  used  as  the  basis  for  a  linear  temporal  logic  model  checking  algorithm.  I  his  algorithm 
has  the  time  complexity  f>((|.V|  +  |K|)- i' F>y  I.icluenstein  and  I’nucli  further  showed  how  this 
basic  algorithm  could  be  extended  to  handle  a  number  of  different  notions  of  fairness  with  essentially 
the  same  complexity. 

An  alternative  approach  due  to  Vardi  and  Wolper  (Vardi  &  Wolper  86]  exploits  the  close  relationship 
between  linear  temporal  logic  formulas  and  Riichi  .lutomata.  A  Ihtcht  automata  is  a  tuple 
A  =  (2,  S.  p.  S0,  F),  where 

•  2  is  an  alphabet. 

•  S  is  a  set  of  states. 

•  p  :  Sx  2  — *  2s  is  a  nondcterministic  transition  function. 

•  5,,C5  is  a  set  of  initial  states. 

•  /  C  S  is  a  set  of  designated  states. 

A  run  of  A  on  an  infinite  word  w  =  ala, ...  is  a  sequence  v0r.  .  . .  where  ,s0 1  Stj  and  5,  t  p(  s,_  .  it.) ,  for 
all  i>  1 .  A  run  . . .  is  accepting  if  there  is  some  designated  state  that  repeals  infinitely  often,  i.e.,  for 
some  5  €  F  there  arc  infinitely  many  i 's  such  that  s,  -  s.  The  infinite  word  *v  is  accepted  by  I  if  there  is 
an  ucccpung  run  of  A  over  w.  The  set  of  infinite  words  accepted  by  .1  is  denoted  L(,-l).  The  following 
theorem  is  proved  in  [Vardi  &  Wolper  86]. 

Lemma  7:  For  every  linear  temporal  formula  A/,  a  Riichi  automata  Aj  can  be  constructed,  where 
2  =  2'^  and  |S|  <2!eni!>^ ,  such  that  L(Aj-)  is  exactly  die  set  of  computations  satisfying  die  formula 
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A  Kripke  Structure  M  -  (.V.  R,  /.)  with  initial  state  r0t.S'can  be  viewed  as  a  Riichi  automaton 
•  1,1/  =  (2.  -V,  {s„}.  p.  S)  where  2  =  lAP  and  s'  €  p(j.  a)  iff  (x.  s')  t  R  and  a  =  !.($).  Note  that  any 
infinite  run  of  this  automaton  is  accepting.  U  fA/)  is  the  set  of  computations  of  Ast.  Phis.  m  order  to 
determine  whether  \f.s¥=  \  /'  it  is  sufficient  tocheck  whether  i(.1u)nf(  t  )  is  empty  lliixe.m  be 


determined  by  an  automata  theoretic  construction  with  essentially  the  same  time  complexity  as  the 
Pnueli  Lichtenstein  algorithm. 


One  of  die  expected  advantages  of  using  linear  temporal  logic  is  that  fairness  constraints  can  be 
handled  directly.  However,  if  fairness  constraints  arc  included  as  part  of  the  specifications,  the  formulas 
that  must  be  checked  will  in  general  be  quite  large.  For  instance,  consider  a  fairness  constraint  which 
requires  that  progress  be  made  from  any  state  in  the  program.  The  formula  that  expresses  this  property 
is 

\[  A  ->G (<j/.y)  -» (.rest  vf  specification)], 
r<= 

which  has  si/e  G(|.V|).  This  problem  was  realized  by  Lichtenstein  and  Pnueli  and  by  Vardi  and  Wolpcr. 
They  in  fact  handle  fairness  by  means  of  fairness  constraints  in  a  manner  very  similar  to  the  way  it  is 
handled  in  (Clarke  ct  al  86a],  Another  problem  with  using  linear  temporal  logic  is  that  in  general  it  is 
impossible  to  handle  specifications  which  involve  existential  path  quantifiers.  Although  it  is  possible  to 
check  simple  formulas  of  the  form  F/ where  f  is  a  restricted  paih  formula,  it  is  not  possible  to  check 
formulas  like  AG(F.F/),  which  is  used  to  express  absence  of  deadlock  in  die  example  in  section  5. 
Moreover,  model  checking  for  the  frill  logic  CTL*  is  no  more  difficult  than  for  linear  temporal  logic  as 
was  shown  by  Emerson  and  Lei  (Emerson  &  Lei  85]. 

Theorem  8:  If  we  arc  given  an  algorithm  A/.f  TL  to  solve  the  model  chocking  problem  for  linear 
temporal  logic,  then  wc  can  construct- an  algorithm  ALctl *  for  the  full  logic  Cl  L  ’  that  has  the  same 
order  of  complexity  as  AL{  TL. 

7.  Applications 

Sequential  circuit  verification  is  a  natural  application  for  die  type  of  verifier  discussed  in  this  paper. 
Bochrnann  (Bochmann  82]  was  probably  the  first  to  realize  the  usefulness  of  temporal  logic  for 
describing  the  behavior  of  circuits.  He  verified  an  implementation  of  a  seif-timed  arbiter  using  linear 
temporal  logic  and  what  he  called  "reachability  analysis."  The  work  of  Malachi  and  Ow  icki  (Malaehi  & 
Owicki  81]  identified  addidonal  temporal  operators  required  to  express  interesting  properties  of  circuits 
and  also  gave  specifications  for  a  large  class  of  modules  used  in  self-timed  circuits.  Although  these 
researchers  contributed  significantly  toward  developing  an  adequate  notation  for  expressing  die 
correctness  of  sequential  circuits,  the  problem  of  mechanically  verifying  a  circuit  remained  unsolved. 

In  [Mishra  &  Claikc  85]  Clarke  and  Mishra  showed  how  die  EMC  algorithm  could  be  used  to  unify 
various  temporal  properties  of  asynchronous  circuits.  They  developed  a  technique  for  extracting  a  Mate 


graph  directly  from  a  wire-list  description  of  the  circuit  (i.c.,  from  a  description  of  the  circuit  in  terms  of 
its  components  and  their  interconnections).  The  model  checker  was  then  used  to  show  that  state  graph 
satisfied  various  specifications  expressed  in  temporal  logic.  In  this  way  they  were  able  to  determine  that 
a  self-timed  queue  element  described  in  Seitz'  chapter  of  Mead  and  Conway  [Seitz  80]  did  not  satisfy  its 
specifications.  Their  work  was  later  extended  by  Browne.  Cl.nke,  Dill,  and  Mishra [Browne  et  al 
86]  who  showed,  in  general,  how  a  mixed  gate  and  switch  level  circuit  simulator  could  be  used  to  extract 
a  state  graph  from  a  structural  description  of  a  sequential  circuit.  The  basic  simulation  algorithm  is 
shown  in  Figure  7-1.  Circuits  are  usually  designed  under  the  assumption  that  certain  input  sequences 
and  combinations  will  not  occur.  1  heir  program  exploits  tins  observation  to  prevent  a  combinatorial 
explosion  in  the  number  of  states  that  are  generated,  by  allowing  the  user  to  specify  a  set  of  conditions 
under  which  die  inputs  can  change. 

{The  procedure  below  uses  a  hash  table  that  maps  node 
value  assignments  to  states. To  construct  the  state  machine, 
call  this  procedure  on  a  node_val ue_ass i gmnent  for  the 
initial  state.} 

procedure  Bu  i  1  dGr  aph  (  Node_val  ue_as  s  i  ynmen  t )  return  a  state 
begin 

if  there  is  a  state  for  the  node_value_assignment 

already  in  the  table  then  return  the  state; 

else 

Create  a  new  state; 

Label  state  with  nodes  that  have  1  values; 

Store  state  and  node  values  together  in  hash  table; 

for  each  possible  input  assignment  do 

Combine  current  values  for  internal  nodes  and  input 
assignment  into  a  new  node_va 1 ue_ass i gnmen t ; 

Simulate  one  step  to  find  a  new  node  assignment; 

Call  BuildGraph  recursively  on  new  node  assignment; 

Add  value  returned  by  previous  line  to  successors  of 
current  state; 

end 

end 

end 

Figure  7-1:  Algorithm  For  Constructing  Kripke  S tincture  F  rom  Circuit 

The  circuit  simulator  in  [Browne  et  al  86]  used  a  unit-delay  timing  model  in  which  the  switching 
delays  of  all  the  transistors  and  gates  arc  assumed  to  be  equal.  While  a  unit-delay  model  is  satisfactory 
for  synchronous  circuits,  it  may  not  be  appropriate  for  asynchionous  circuits.  In  [Dill  &  Clarke  S6|  1  hll 
and  Clarke  showed  how  Kripke  structures  could  be  extracted  from  a  gate  level  description  of  a  circuit 
under  a  model  of  circuit  behavior  that  permitted  arbitrary  nou-/cro  delays  to  be  associated  with  the 
outputs  of  the  gates.  Fhe  basic  idea  behind  their  approach  is  quite  simple.  Consider  an  WD  vie  with 


two  inputs,  x  and>\  and  a  single  output  z.  Assume  that  the  gate  is  in  an  unstable  configuration  with  x 
low,  y  high,  and  z  high.  The  Kripke  structure  for  the  circuit  containing  this  gate  will  have  a  state 
corresponding  to  the  unstable  configuration  as  shown  in  Figure  7-2.  The  state  will  have  a  self-loop  and 
a  transition  to  another  state  representing  a  stable  configuration  in  which  the  output  is  low.  Fairness 
constraints,  as  described  in  Section  4.  arc  used  to  insure  that  die  system  doesn't  remain  in  an  unstable 
configuration  forever.  In  die  case  of  the  AND  gate,  it  is  sufficient  to  require  diat  infinitely  often 
z  =  xAy. 


Figure  7-2:  Krikpc  structure  for  unstable  configuration  of  AND  gate. 

In  practice,  die  arbitrary  delay  model  is  much  too  conservative.  Many  circuits  arc  ‘almost  speed 
independent":  They  do  not  appear  to  be  correct  under  a  pure  arbitrary  delay  model,  but  would  work 
given  reasonable  assumptions  about  the  relationships  between  die  delays.  When  the  draft  designer 
has  a  great  deal  of  control  over  the  magnitudes  of  circuit  delays,  exploiting  more  detailed  knowledge  of 
circuit  timing  can  result  in  smaller  and  faster  circuits.  In  fact,  actual  circuits  often  rely  on  such 
assumptions.  In  [Browne  et  al  85]  and  [Dill  86]  a  mcdiod  is  described  for  adding  such  assumptions  to  a 
circuit  description  and  incorporating  them  into  the  state-graph  construction.  Possible  timing  constraints 
include  constant  upper  and  lower  bounds  on  individual  delays,  and  bounds  on  die  differences  between 
delays.  Using  constraints  of  this  form,  one  can  say  for  example:  “the  delay  of  the  first  AND  gate  is 
between  5  and  10  nanoseconds”  or  "the  delay  of  the  first  AND  gate  is  greater  dian  the  delay  of  die 
second  AND  gate."  The  state  graph  constructed  with  respect  to  a  particular  set  of  delay  assumptions 
rules  out  some  circuit  executions  which  would  be  aliowed  under  an  arbitrary  delay  model.  Ilcr.ee, 
formulas  in  CTL  which  might  not  have  been  time  in  an  arbitrary  delay  model  may  be  true  with  respect 
to  particular  delay  assumptions  (because  all  the  counterexample  padis  are  ruled  out  by  the  deity 
assumptions).  This  technique  was  applied  to  a  patented  asynchronous  queue  cell  m  [Hmwne  et  al  85). 
The  authors  determined  that  the  circuit  did  not  meet  its  specifications  under  the  arbitrary  element  delay 
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model.  However,  under  the  assumption  that  the  input  was  slower  than  two  of  the  circuit  gates,  they 
showed  that  the  circuit  met  its  temporal  logic  specifications. 

An  alternative  approach  obtains  the  state  diagram  by  compilation  from  a  specification  of  the  original 
(synchronous)  circuit  in  a  simple  programming  language-like  notation.  Browne  and  Clarke  (  (Browne  et 
al  S6J,  (Browne  &  Clarke  86))  use  a  Pascal-like  state  machine  description  language  called  SMI.  for  this 
purpose.  I  he  language  includes  the  standard  control  structures  if,  while,  and  loop/cxit.  A  collegia 
statement  is  also  provided  for  simultaneous  execution  of  statements  in  lock-step.  Since  SMI.  programs 
will  ultimately  be  implemented  in  hardware,  the  only  data  types  permitted  arc  boolean  and  (bounded) 
integer.  The  output  of  tire  SML  compiler  is  a  deterministic  Moore  Machine  that  can  be  automatically 
implemented  as  a  PI  A,  PAL,  or  a  ROM.  The  output  can  also  be  analyzed  for  correctness  using  the 
EMC  algorithm.  In  [Brow  ne  86]  Browne  describes  a  specialized  version  of  the  EMC  algorithm  that  can 
check  Moore  machines  much  more  rapidly  than  the  original  algorithm. 

Another  potential  area  of  application  is  the  verification  of  network  communication  protocols.  The 
alternating  bit  protocol  [Bartlet  et  al  69]  for  reliable  transmission  of  messages  by  a  noisy  communication 
channel  is  a  simple  example  of  such  an  algorithm.  By  using  the  CTL  model  checking  procedure  it  is 
possible  to  determine  in  a  few  seconds  whether  this  protocol  meets  its  specifications  [Clarke  ct  al  86a). 
Sifakis  at  Grenoble  [Quiellc  &  Sifakis  81]  and  Kurshan  at  Bell  l  abs  [Kurshan  86]  have  also  considered 
applications  involving  network  protocols.  The  delay  assumptions  mentioned  above  may  be  useful  for 
describing  the  real-time  behavior  of  such  protocols. 

8.  Conclusion 

Although  the  verification  technique  described  in  this  paper  has  already  been  used  to  find  some 
nontrivial  errors  in  circuit  designs  and  communications  protocols,  more  research  needs  to  be  done 
before  it  will  become  a  truly  practical  debugging  tool  for  use  by  system  dcsigneis.  One  problem  is  the 
expressibility  of  the  underlying  temporal  logic.  For  circuit  specification  twang  Jiagraws  may  be  more 
natural  to  use  than  temporal  logic  formulas.  Of  course,  temporal  logic  is  more  general  since  there  is  no 
analogue  of  negation,  disjunction,  or  conjunction  for  timing  diagrams.  It  may  be  possible  to  either 
systematically  translate  timing  diagrams  into  temporal  logic  formulas  or  check  them  directly  using  an 
algorithm  similar  to  the  one  used  by  the  model  checker.  If  so,  tins  would  simplify  the  task  of  specifying 
a  complicated  circuit  and  also  allow  the  designer  to  be  moie  confident  that  specifications  actually  mean 
what  he  dunks  they  mean. 
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The  most  important  problem,  however,  is  the  v late  explosion  problem.  There  arc  several  different 
strategics  for  handling  this  problem.  In  verifying  asynchronous  circuits,  for  example,  buggy  circuits 
sometimes  result  in  much  larger  state  graphs  than  correct  circuits.  This  happens  because  the  activity  in 
tire  circuit  is  much  more  disordered  after  an  error  has  occutrcd.  One  possible  solution  in  this  case  is  to 
run  the  program  which  build',  the  state  graph  and  the  model  checker  as  co-routines,  creating  states  only 
as  they  need  to  be  referenced  by  the  model-checker.  In  [Dill  86]  this  technique  is  called  lazy  state 
generation,  by  analogy  to  lazy  evaluation  in  programming  language  implementations.  By  using  this 
method,  an  error  could  be  discovered  and  reported  after  constructing  only  a  small  pan  of  the  entire 
state  graph;  tins  would  not  only  speed  up  the  verification  process,  it  would  also  make  it  pos>.h!c  to 
verify  some  circuits  which  could  not  be  verified  if  the  entire  graph  had  to  be  constructed. 

Another  approach  to  the  suite  explosion  problem  is  to  exploit  the  hierarchical  structure  of  complex 
finite  state  concurrent  systems.  If  an  appropriate  subset  of  Cl  I .  is  used!  (Mishra  &  Clarke  85],  [Clarke 
et  al  86b]),  dien  lower  level  subcircuits  can  be  simplified  by  "hiding"  some  of  their  internal  nodes  (more 
precisely,  making  it  illegal  to  use  diem  in  temporal  logic  formulas)  and  merging  groups  of  states  that 
become  indistinguishable  into  single  state.  Preliminary  research  in  [Mishra  &  Clarke  85]  indicates  that 
by  using  this  technique  it  may  be  possible  to  cut-down  dramatically  on  the  number  of  stales  that  need 
to  be  examined. 

Finally,  special  techniques  may  be  appropriate  for  concurrent  systems  that  are  composed  of  many 
identical  processes.  Consider,  for  example,  a  distributed  mutual  exclusion  algorithm  for  pucesscs 
arranged  in  a  ring  network  in  which  mutual  exclusion  is  guaranteed  by  means  of  a  token  that  is  passed 
around  the  ring  ( [Dijkstra  85],  [Kurshan  85],  [Martin  85]).  A  strategy  that  is  often  used  for  debugging 
such  systems  is  to  consider  first  a  reduced  system  with  one  or  two  processes.  If  it  is  possible  to  'how  that 
the  reduced  system  is  correct  and  if  the  individual  processes  are  really  identical,  then  one  is  tempted  to 
conclude  diat  die  entire  system  will  be  euiicct.  lufCluikc  et  al  Gob]  an  attempt  is  made  to  |:-r* • » ,elo  a 
solid  theoretical  basis  that  will  prevent  fallacious  conclusions  in  arguments  of  this  type.  I  lie  authors 
describe  a  temporal  logic  called  Indexed  CT!  ,  or  ICTI.  for  specifying  networks  of  identical  processes. 
Ihc  logic  includes  all  of  CM.*  with  die  exception  of  the  nexttime  operator;  in  addition,  it  permits 
formulas  of  the  form  A  /(/)nnd  V  y  ( r )  where  /(/)  is  a  formula  in  which  all  of  the  atomic  propositions 

l  l 

arc  subsciipted  by  i.  A  Kripkc  structure  tor  a  family  of  ,V  identical  processes  may  be  obtained  as  a 
product  of  the  suite  graphs  of  the  individual  processes.  Instances  of  the  same  atomic  po. position  in 
different  processes  are  distinguished  by  using  the  number  of  the  process  as  a  subscuni.  thus,  I, 
represents  the  instance  ol  atomic  pioposition  I  .I'sociated  with  process  5. 


Since  a  closed  formula  of  the  new  logic  cannot  contain  any  atomic  propositions  with  constant  index 
values,  it  is  impossible  to  refer  to  a  specific  process  by  writing  such  a  formula.  Hence,  changing  the 
number  of  processes  in  a  family  of  identical  processes  should  not  effect  the  truth  of  a  formula  in  the 
logic.  This  intuitive  idea  is  made  precise  by  introducing  a  new  notion  of  bisimulation  [Milner 
79]  between  two  Kripke  structures  with  the  same  set  of  indexed  propositions  hut  different  sets  of  index 
values.  It  is  possible  to  prove  that  if  two  structures  correspond  in  this  manner,  a  closed  formula  of 
Indexed  CTL*  will  be  true  in  the  initial  state  of  one  if  and  only  if  it  is  true  in  the  initial  state  of  the 
other. 

These  ideas  arc  illustrated  in  [Clarke  ct  al  86b]  by  considering  the  distributed  mutual  exclusion 
algorithm  mentioned  above.  The  atomic  proposition  c,  is  true  when  the  /-th  process  is  in  its  critical 
region,  and  the  atomic  proposition  d(  is  true  when  the  /-th  process  is  delayed  waiting  to  enter  its  critical 
region.  A  typical  requirement  for  such  a  system  is  that  a  process  waiting  to  enter  its  critical  region  will 

eventually  do  so.  This  condition  is  easily  expressed  in  ICTL*  by  the  formula  A  AG(<i,=>  AFc,).  The 

.  ' 

results  of  [Clarke  ct  al  86b]  can  be  used  to  show  that  exactly  the  same  ICTL  formulas  hold  in  a  network 
with  1000  processes  as  hold  in  a  network  with  two  processes.  The  EMC  algorithm  can  be  used  to  check 
automatically  that  tire  above  formula  holds  in  networks  of  sizc-two  and  conclude  that  ic  will  also  hold  in 
networks  of  size  1000.  At  present  this  methodology  has  only  been  partially  automated,  however.  The 
bisimulation  must  be  established  by  hand  and  this  generally  requires  some  representation  of  the  larger 
Kripke  structure.  Several  researchers  arc  attempting  to  fmd  a  way  of  automating  this  phase  in  a  manner 
that  avoids  building  the  larger  Kripke  structure. 

Other  techniques  for  avoiding  the  state  explosion  problem  arc  being  investigated  by  Kurshan  and 
Wolpcr.  In  Kurshan’s  system  [Kurshan  85]  this  problem  is  handled  by  using  a  homomorphism  to 
collapse  a  large  state  machine  into  a  much  smaller  one  while  preserving  those  properties  that  arc 
iinpodant  for  verification.  Since  Kurshan  does  not  use  temporal  logic  formulas  for  spcciticatmn,  he  has 
no  analogue  of  the  indexed  formulas  or  of  the  bisimulation  theorem  used  in  [Clarke  ct  al  86b],  Wolpcr 
[Wolpcr  86]  considers  a  logic  somewhat  like  ICTL*  for  reasoning  about  programs  that  arc  d.ita- 
independent;  however,  his  indexed  variables  range  over  data  elements,  not  over  processes.  Also,  there 
is  no  notion  of  correspondence  between  structures  in  his  work.  Some  ultimate  limitations  on  this  type  of 
reasoning  arc  discussed  in  Apt  and  Kozcn  [Apt  &  Kozcn  86]. 
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Research  On  Automatic  Verification 
Of  Finite-State  Concurrent  Systems 


E.  M.  Clarke  and  0.  Cilimberg 
Carnegie  Mellon  University,  Pittsburgh 

1 .  Introduction 

Temporal  logics  were  first  developed  by  philosophers  for  reasoning  about  the  ordering  of  events  in 
time  without  introducing  time  explicitly  [Hughes  &  Creswell  77],  Although  a  number  of  different 
temporal  logics  have  been  studied,  most  have  an  operator  like  G  (/)  that  is  true  in  the  present  if  /is 
always  true  in  the  future  (Le.,  if  /is  globally  true).  To  assert  that  two  events  el  and  e2  never  occur  at  the 
same  time,  one  would  write  G(-'e1v-ie2).  Temporal  logics  are  often  classified  according  to  whether 
time  is  assumed  to  have  a  linear  or  a  branching  structure.  This  classification  may  occasionally  be 
misleading  since  some  temporal  logics  combine  both  linear-time  and  branching-time  operators. 
Instead,  we  will  adopt  the  approach  used  in  [Emerson  &.  Halpern  83]  dtat  permits  both  types  of  logics  to 
be  treated  within  a  single  semantical  framework.  In  this  papefthe  meaning  of  a  temporal  logic  formula 
will  always  be  determined  with  respect  to  a  labelled  state  transition  graph;  for  historical  reasons  such 
structures  arc  called  Kripke  models  [Hughes  &  Creswell  77], 

Pnueli  was  apparently  the  first  person  to  use  temporal  logic  for  specifying  and  verifying  concurrent 
programs  [Pnculi  77],  His  approach  involved  proving  desired  properties  of  the  program  under 
consideration  from  a  set  of  program  axioms  that  described  the  behavior  of  the  individual  statements  in 
the  program.  Proofs  were  usually  constructed  by  hand,  and  this  task  was  in  general  quite  tedious.  Since 
many  concurrent  programs  can  be  viewed  as  communicating  finite  state  machines,  there  was  a  strong 
possibility  that  at  least  some  of  these  programs  could  be  automatically  verified.  T  he  first  verification 
technique  10  exploit  this  observation  was  the  CTl.  model  checking  procedure  developed  by  Claike  and 


Emerson  in  [Clarke  &  Emerson  81].  Their  algorithm  was  polynomial  in  both  the  sire  of  the  model 
determined  by  the  program  under  consideration  and  in  the  length  of  its  specification  in  temporal  logic. 
They  also  showed  how  fairness  [Gabbay  et  .*1  80]  could  be  handled  without  changing  the  complexity  of 
•their  algorithm.  Handling  fairness  was  an  important  step  since  the  correctness  of  many  concurrent 
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algorithms  depends  critically  on  some  assumption  of  this  type;  for  example,  absence  of  starvation  in  a 
mutual  exclusion  algorithm  may  depend  on  the  assumption  that  each  process  makes  progress  infinitely 
often. 

At  roughly  the  same  time  Quiellc  and  Sifakis  [Quiche  &  Sifakis  81]  gave  a  model  checking  algorithm 
for  a  similar  branching-time  logic,  but  they  did  not  analyze  its  complexity  or  show  how  t»  handle  an 
interesting  notion  of  fairness.  Later  Clarke,  Emerson,  and  Sistla  [Clarke  ct  a!  86a]  devised  an  improved 
algorithm  that  was  linear  in  the  product  of  the  length  of  the  formula  and  in  the  size  of  the  global  state 
graph.  Sistla  and  Clarke  [Sistla  &  Clarke  86]  analyzed  the  model  checking  problem  for  a  variety  of  other 
temporal  logics  and  showed,  in  particular,  that  for  linear  temporal  logic  the  problem  was  PSPACE 
complete. 

A  number  of  papers  have  shown  how  the  temporal  logic  model  checking  procedure  can  be  used  for 
verifying  network  protocols  and  sequential  circuits  ( [Clarke  et  al  86a],  (Mishra  &  Clarke  85J,  [Browne  et 
al  86],  [Dill  &  Clarke  86],  [Browne  ct  al  85],  [Browne  &  Clarke  86],  [Browne  et  al  6b]).  In  the  ease  of 
sequential  circuits  two  approaches  have  been  developed  for  obtaining  state  transition  graphs  to  analyze. 
The  first  approach  extracts  a  stale  graph  directly  from  the  circuit  under  an  appropriate  timing  mode!  of 
circuit  behavior.  The  second  approach  obtains  a  state  transition  graph  by  compilation  from  a  high  level 
representation  of  the  circuit  in  a  Pascal-like  programming  language.  In  practice  the  model  checking 
procedure  is  able  to  check  state  transition  graphs  at  a  rate  of  100  states  per  second  for  formulas  of 
reasonable  length.  It  has  been  used  successfully  to  find  previously  unknown  errors  in  published  designs 
of  asynchronous  circuits. 

Alternative  approaches  have  been  proposed  by  a  number  of  other  researchers.  The  appioach  used  by 
Kurshan  [Kurshan  86]  involves  checking  inclusion  between  two  automata  on  infinite  tapes.  The  first 
machine  represents  the  system  that  is  being  verified;  tine  second  represents  its  specification.  Automata 
on  infinite  tapes  arc  used  in  order  to  handle  fairness.  Pnucli  and  Lichtenstein  [Lichtenstein  &  Pnueli 
85]  reanalyzed  the  complexity  of  checking  linear-time  formulas  and  discovered  that  although  the 
complexity  appears  exponential  in  the  length  of  the  formula,  it  is  linear  in  the  m/c  of  the  global  state 
graph.  Based  on  this  observation,  they  argued  that  the  high  complexity  of  linear-time  model  checking 
might  still  be  acceptable  for  short  formulas.  Emerson  and  Lei  [Emerson  &  Lei  85]  extended  their  result 
to  show  that  formulas  of  die  logic  CTL*.  which  combines  both  branching-time  and  linear-time 
operators,  could  be  checked  with  essentially  the  same  complexity  as  formulas  of  linear  temporal  logic. 
Vjrdi  and  W'olpcr  have  recently  [Vardi  &  Wolpor  86]  shown  how  the  model  checking  problem  can  be 
formulated  in  terms  of  automata,  thus  relating  the  model  checking  apppuJi  io  the  woik  of  KuNun. 
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Although  the  model  chocking  procedure  discussed  in  this  paper  has  already  been  used  to  discover 
some  surprising  errors  in  non-trivial  programs,  more  work  still  remains  to  be  done.  Certainly  the  most 
serious  problem  is  the  stale  explosion  problem.  In  analyzing  a  system  of  ,V  processes,  the  number  of 
states  in  the  global  state  graph  may  grow  exponentially  with  N.  Recent  research  indicates,  however, 
that  it  may  be  possible  to  avoid  this  prohlem  in  some  important  eases.  For  instance,  techniques 
developed  in  (Clarke  ct  al  86b]  may  reduce  the  size  of  the  state  graph  that  needs  to  be  searched  when 
many  of  the  processes  arc  identical.  It  may  also  be  possible  to  exploit  the  hierarchical  structure  of  a 
complex  concurrent  program  in  order  to  reduce  die  number  of  states  that  need  to  be  considered  at  any 
one  level  of  abstraction  [Mislira  &  Clarke  85], 

This  survey  is  organized  as  follows:  Section  2  describes  the  syntax  and  semantics  of  the  temporal 
logics  that  arc  used  in  this  paper.  In  Section  3  we  state  the  model  checking  problem  and  give  an  efficient 
algorithm  for  checking  simple  branching-time  formulas.  In  Section  4  we  discuss  die  issue  of  fairness 
and  show  how  the  algoridim  of  Section  3  can  be  extended  to  include  fairness  constraints.  Section  5 
demonstrates  how  the  model  checking  algoridim  can  be  used  to  debug  a  simple  mutual  exclusion 
program.  In  Section  6  we  describe  some  alternative  approaches  for  verifying  systems  of  finite  state 
concurrent  processes.  We  analyze  the  complexity  of  checking  linear  temporal  logic  formulas  and  outline 
die  techniques  of  Pnucli  and  Lichtenstein  [Lichtenstein  &  Pnucli  85)  and  Vardi  and  Wolpcr(Vardi  & 
Wolper  86]. Additional  applications  to  circuit  and  protocol  verification  arc  discussed  in  Section  7.  The 
paper  concludes  in  Scctiou  8  with  a  discussion  of  some  of  the  important  remaining  research  problems 
like  the  stale  explosion  problem. 

2.  Computation  Tree  Logics 

In  this  paper  finite  state  programs  arc  modelled  by  labelled  state-transition  graphs,  called  Kripke 
structures  [Hughes  &  Crcswcll  77],  If  some  state  is  designated  as  the  initial  state,  then  the  Kripke 
structure  can  be  unwound  into  an  infinite  tree  with  that  suite  as  the  root.  Sir.ee  paths  in  the  tree 
represent  possible  computations  of  the  program,  we  will  refer  to  die  infinite  tree  obtained  in  this 
manner  as  the  computation  tree  of  the  program.  Temporal  logics  may  differ  according  to  bow  they 
handle  branching  in  the  underlying  computation  tree.  In  linear  temporal  logic,  operators  arc  punided 

for  describing  events  along  a  single  computation  path.  In  a  branching-time  logic  die  temporal  operators 

* 

quantify  over  the  paths  that  are  possible  from  a  given  state.  The  computation  tree  logic  ('ll 
( [Lmerson  &  Clarke  81],  (F.mcrson  &  Halpcrn  81],  [Clarke  ct  al  86a])  combines  both  branching-tune 
and  linear-time  operators:  a  path  quantifier,  either  A  ("for  all  computation  piths")  or  L  ("for  -mne 
computation  path")  can  prefix  an  assertion  tompoH'd  of  arhitr.n  v  combinations  of  die  usual  line  u  tune 


operators  G  ("always"),  F  ("sometimes").  X  ("nexttime"),  and  U  ("until").  The  remainder  of  this 
section  gives  a  precise  description  of  the  syntax  and  semantics  of  these  logics. 

There  are  two  types  of  formulas  in  CT1.  :  state  formulas  (which  are  uuc  in  a  specific  state)  and  path 
formulas  (which  arc  true  along  a  specific  path).  Let  AP  be  the  set  of  atomic  proposition  names.  A*statc 
formula  is  either: 

•  A,  if  AeAP. 

•  If  /  and  g  are  state  formulas,  then  -> /  and  /  v  g  are  state  formulas. 

•  If  /  is  a  path  formula,  then  K(/)  is  a  state  formula. 

A  path  formula  is  either: 

•  A  state  formula. 

•  If  /  and  g  are  path  formulas,  then  ->/  /  v  g ,  X/  and / U  g  are  path  formulas. 

CTL*  is  the  set  of  state  formulas  generated  by  the  above  rules. 

CTL  ( [Ben-An  et  al  83],  [Clarke  &  Emerson  81])  is  a  restricted  subset  of  CrL*  that  permits  only 
branching-time  opcratorv-each  path  quantifier  must  be  immediately  followed  by  exactly  one  of  the 
operators  G,  F,  X,  or  U.  More  precisely,  CTL  is  the  subset  of  CTL’  that  is  obtained  if  the  path  formulas 
arc  restricted  as  follows: 

•  If  /  and  g  are  state  formulas,  then  X / and  /V  g  are  path  formulas. 

•  If  /  is  a  path  formula,  then  so  is  ->/ 

Linear  temporal  logic  (LTL),  on  the  other  hand,  will  consist  of  formulas  that  have  the  form  A /  where 
/  is  a  path  formula  in  which  the  only  state  subformulas  that  are  permitted  are  atomic  propositions. 
More  formally,  a  path  formula  is  either 

•  An  atomic  proposition 

•  If  /  and  g  arc  path  formulas,  then  ->f  /  v  g ,  Xf  and/U  g  are  path  formulas. 

We  define  the  semantics  of  CTL*  with  respect  to  a  structure  M = <S,  R,  L>,  where 

•  5  is  a  set  of  states. 

•  ROSxS  is  the  transition  relation,  which  must  be  total.  We  write  s,  — *  s,  to  indicate  that 

(S1,5j)€  /?. 

•  L:  S— *9(,4P)  is  a  function  that  labels  each  state  with  a  set  of  atomic  propositions  true  in 


5 


Unless  otherwise  stated,  all  of  our  results  apply  only  to  finite  Kripkc  structures. 

We  define  a  path  in  M  to  be  a  sequence  of  states,  n  =  s^.s. _ such  that  for  every />0,  j,  -*  j(>1.  it' 

will  denote  the  suffix  of  v  starting  at  s,. 

We  use  the  standard  notation  to  indicate  that  a  state  formula  f  holds  in  a  structure:  A/,jl=  /means 
that  /  holds  at  suite  s  in  structure  A/.  Similarly,  if  /  is  a  path  formula,  \I,tt  1=  /  means  that  /  holds 

along  path  it  in  structure  A/,  (  he  relation  1=  is  defined  inductively  as  follows  (assuming  that  /  and/ 

are  state  formulas  ar.d  g ,  and  g  2  are  path  formulas): 

1.  iN»/f  » 

2.  jN  ->fi  «=»  sb48/. 

3. 5N/v/  «  jN/orsl=/j. 

4.  sl=  K(gt)  <=>  there  exists  a  path  v  starting  with  s  such  that  w  1=  gv 

5. 7r  h=  /'  «=»  5  is  the  first  state  of  v  and  sk=/. 

6.  w  -igj  =>  v  h*  gj. 

7.  it  \=  v  =>  7T  gv  or  n  N  gv 

8.  rr  Xgt  <=»  ir1 1= 

9.  wN^Ugj  «=>  there  exists  a  k  >  0  such  that  irkt=gl  and  for  all  0  <j  <  k,  7/1=  g,. 

We  will  also  use  the  following  abbreviations  in  writing  CTL*  (CTL  and  1 .11.)  formulas: 


•/Ag  =  V  ~>g) 


•  A(/)  S  -K(-V) 


•  F/  =  true  U / 


•  G /  =  — ■  F— >/. 


In  ( [Lamport  80],  [Emerson  &  Halpcrn  83])  it  is  shown  that  the  three  logics  discussed  in  this  section 
have  different  expressive  powers.  Tor  example,  there  is  no  CH.  formula  that  is  equivalent  to  the  I.TL 
formula  ,1(FG/>).  Likewise,  there  is  no  Lll.  formula  that  is  equivalent  to  the  CTL  formula 
AG(  EFp).  The  disjunction  of  these  two  formulas  l(  FG  p)  v  AG(  LK/; )  is  a  CTL*  formula  that  is  not 
expressible  in  cither  CLL  or  LTL, 

3.  The  CTL  Model  Checking  Algorithm 

I  et  A/  =  (S,  R,  /.)  be  a  finite  Kripke  •.tructuic.  Assume  that  we  want  to  determine  which  suites  in  .V 
satisfy  die  Cl  L  formula  />.  We  will  design  t.ur  algorithm  to  operate  m  stages;  I  he  first  stage  privesses 
all  subforimil.is  of  /  of  lencih  1.  the  second  stage  processes  all  suMonrail.i .  .a  'ength  ?.  .aid  ,>n.  \t 


6 


the  end  of  the  i,h  stage,  each  suite  will  be  labeled  with  the  set  of  all  subformulas  of  length  less  than  or 
equal  to  i  that  are  true  in  the  state.  We  let  the  expression  labe!(s )  denote  this  set  for  state  5.  When  the 
algorithm  terminates  at  the  end  of  stage  n  =  length(fD),  we  see  that  for  all  states  and  for  all 
subformulas /of /,  \(.s  N  /iff  /e  label(s)  . 

Observe  that  AX  can  be  expressed  in  terms  of  F.X  and  that  AU  can  be  expressed  in  terms  of  FU  and 
EG: 

AX/--KX-/ 

A U/]  =  -(K[-/  U (-/  A  -./)]  v  EG( ->/)). 

1  hus,  for  the  stage  1  algorithm  it  is  sufficient  to  be  able  to  handle  six  cases,  depending  on  w  hetlier  /  is 
atomic  or  has  one  of  the  following  forms:  ->/,  /  v /,  F.X/,  F(/  U /]  or  F.G /. 

We  will  only  consider  die  last  two  eases,  since  the  others  arc  suaightforward. 

To  handle  formulas  of  the  form  /=  F.(/U/l  we  first  find  all  of  those  suites  that  arc  labeled  with  /. 
We  then  work  backwards  using  die  converse  of  die  transition  -relation  R  and  find  all  of  those  states  due 
can  be  reached  by  a  path  in  which  each  state  is  labeled  with  /.  All  such  states  should  be  labeled  with  /. 
This  step  requires  time  0(\ ,S|  + 1  R  | ). 

The  ease  in  which  /=  FG /  is  slightly  more  complicated  and  depends  on  the  following  observation. 

I.entma  1:  Let  M'  be  obtained  from  M  by  deleting  from  S  all  of  diose  states  at  which  /  does  not 
hold  and  restricting  R  and  /,  accordingly.  Thus,  M'  -  (S' ,  R'.  L')  where  S'  =  {sc  ,S'|.U,sT=/  \ , 
R'  =  ^I5'x5',and  L'  -  /.|y  .  Then  ,l/,5  t=  FG/  iff  the  following  two  conditions  are  satisfied: 

1.  sc-S' 

2.  thee  exists  a  path  in  S'  that  leads  from  3  to  some  node  1  in  a  non-inuu!  strongly  connected 
component2  of  die  graph  (S'  ,R'). 

Proof:  Assume  that  M  s  FG/  .  Clearly  se  S'  .  1  et  w  be  an  infinite  path  suiting  at  r  such  that  / 
holds  at  each  state  on  v  ,  Since  M  is  finite,  it  must  be  possible  to  write  v  as  v  ~  v  where  n .  is  a 
finite  initial  segment  and  w,  is  an  infinite  suffix  of  n  with  die  property  that  c.wh  state  on  rt  -cans 


j* 


,  'M  .*  ■»  Jk'  »  t'l 


.JV  * 


infinitely  often.  Obviously  tr0  is  contained  in  S' .  Let  C  be  the  set  of  states  in  it,.  C  is  a  nontrivial 
strongly  connected  component  of  S' .  To  see  this,  let  ^  and  j2  be  states  in  C.  Pick  some  instance  of  j, 
on  7 r, .  By  the  way  in  which  tt,  was  selected,  we  know  that  there  is  an  instance  of  s2  further  along  it, . 
The  segment  from  5,  to  s2  lies  entirely  within  C  and  hence  within  S' .  This  segment  is  a  finite  path 
from  5,  to  s,  in  S' .  Thus,  both  condition  (1)  and  condition  (2)  arc  satisfied. 

Next,  assume  that  conditions  (1)  and  (2)  are  satisfied.  Let  7 r;  be  the  path  from  s  to  /.  l  et  irl  be  a 
finite  pa  til  of  length  at  least  one  that  leads  from  t  back  to  t.  I  lie  existence  of  it,  is  guaranteed  since  C  is 
a  non-trivial  strongly  connected  component.  All  of  the  states  on  the  infinite  path  it  =  it  ir.u  satisfy  / . 
Since  7t  is  also  a  possible  path  starting  at  s  in  M,  we  see  that  M,s  N  V.Gf .  □ 

The  algorithm  for  the  ease  of  /=  Y.Gf  follows  directly  from  the  lemma.  We  construct  the  restricted 
Kripke  structure  M‘  =(.V/,  R',  L')  as  described  in  the  statement  of  the  lemma.  We  partition  the 
graph  ( S',R ')  into  strongly  connected  components  and  find  those  states  that  belong  to  nontrivial 
components.  We  then  work  backwards  using  the  converse  of  R  and  find  all  of  those  states  that  can  be 
reached  by  a  path  in  which  each  state  is  labeled  with  f .  This  step  also  requires  time  (){ \  S |  +  |  R  \ ). 

In  order  to  handle  an  arbitrary  CTL  formula  f.  we  successively  apply  the  state  labeling  algorithm  to 
die  subformulas  of  f0,  starting  with  the  shortest,  most  deeply  nested  and  work  outward  to  include  all  of 
fn.  Since  each  pass  takes  umc  0(|2>|  4-  1  R\ )  and  since  f  has  lengih(f)  different  subformulas,  die 
entire  algorithm  requires  O(length(f0)  -  (  |  S\  +  |  /?| )). 

lhcorem  2:  [here  is  an  algorithm  for  determining  whether  a  CTL  formula  f0  is  true  in  state  x  of  the 
structure  M  =  (S.  R,  L)  that  runs  in  time  0(length(f0)-  (\S\  -f  | /?!)). 

4.  Fairness  Constraints 

In  verifying  concurrent  systems,  we  are  occasionally  interested  only  in  correctness  along  fair 
execution  sequences.  For  example,  with  a  system  of  concurrent  processes  we  may  wish  to  consider  only 
those  computation  sequences  in  which  each  process  is  executed  infinitely  often.  When  dealing  with 
network  protocols  where  processes  communicate  over  an  imperfect  (or  lossy)  channel  we  may  also  wish 
to  restrict  the  set  of  computation  sequences;  in  this  case  die  unfair  execution  sequences  are  those  in 
which  a  sender  process  continuously  transmits  messages  without  any  reaching  the  receiver  due  to  erratic 
behavior  by  die  channel. 

Uini  ’hlv  qv.ikim-  a  fairness  condition  issoils  ih.it  ivi|tioMS  for  service  aie  c:  ailed  "  aiffi.  icnd  •>!'’  u" 
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Different  concepts  of  what  constitutes  a  "request"  and  what  "sufficiently  often"  should  mean  give  rise 
to  a  variety  of  notions  of  fairness.  Indeed,  many  different  types  of  fairness  and  approaches  to  dealing 
with  them  have  been  proposed  in  the  literature:  we  refer  the  reader  to  [Gahbay  ct  at  SO],  [1  amport  80|. 
(Quiclle  &  Sifakis  82],  and  (l  ehmann  ct  al  SI]  for  more  extensive  treatments.  1'he  text  by  Frances 
[France/  86]  also  gives  an  excellent  survey  of  the  various  ty  pes  of  fairness. 

In  this  section  we  will  show  how  to  extend  the  CTL  model  checking  algorithm  to  handle  a  simple  but 
fundamental  type  of  fairness  in  which  ceitain  predicates  must  hold  infinitely  often  along  every  t  .nr  path. 
( [Clarke  et  al  86a]  shows  how  to  handle  a  richer  class  of  fairness  constraints.)  In  th:s  case  it  follows  from 
[F.mcrson  &  Halpcrn  83]  that  correctness  of  fair  executions  cannot  be  expressed  in  CM .. 

In  order  to  handle  fairness  and  sull  obtain  an  efficient  model  checking  algorithm  we  modify  the 
semantics  of  CIL.  The  new  logic,  which  we  call  CTL  ^  .  has  the  same  sy  max  as  Cl  I..  Hut  a  structure  is 
now  a  4-tuple  ,\f  =  (5,  R,  L,  F)  where  .S',  R,  /.  have  the  same  meaning  as  in  the  case  of  CTL,  and  F  is  a 

c- 

collection  of  predicates  on  5,  F C2  .  A  path  it  is  F-fair  iff  the  following  condition  holds:  for  each 
0  €  F,  there  are  infinitely  many  states  on  n  which  satisfy  predicate  G.  C'l  l/  has  exactly  the  same 
semantics  as  CTL  except  that  all  path  quantifieis  range  over  fair  paths.  The  first  step  in  checking  C  l  [/ 
formulas  is  to  determine  the  fair  strongly  connected  components  of  the  graph  of  \l.  A  strongly  connected 

component  is  fair  if  it  contains  at  least  one  state  from  each  set  in  F.  Formally,  let  F  =  { G . GK  \  be  a 

collection  of  subsets  of  ,V.  A  strongly  connected  component  C  of  the  graph  of  M  ;s  fair  iff  for  each  G,  in 
F ,  there  is  a  state  /,  e  (CD  (7,). 


Lemma  3:  Given  any  finite  structure  M  -  (.S’.  R.  L.  F)  where  F  is  a  set  of  fairness  constraints  and  a 
state  e  5,  the  following  two  conditions  arc  equivalent: 

1.  There  exists  an  F-fair  path  in  M  starting  at  $>. 

2.  There  exists  a  fair  strongly  connected  component  (  of  (the  graph  oi)  \l  sud  that  theio  is  a 
finite  path  from  Sg  to  a  state  i  e  C. 

The  proof  is  straightforward  and  is  given  m  (Clarke  el  al  86a).  We  next  extend  our  model  checking 
algorithm  to  CTL  F .  We  introduce  an  additional  proposition  Q.  which  is  true  at  a  state  iff  there  is  a  fair 
path  starting  from  that  state.  This  can  easily  be  done,  by  obtaining  the  stiongly  connected  components 
of  the  graph  associated  with  die  d  rue  lure  and  marking  a  component  ,.s  fur  if  it  contains  at  least  one 
state  from  each  (7,  in  I',  lly  die  above  lemma  every  state  in  a  fair  strong!-.  connected  component  is  the 
start  of  an  infinite  fair  padi.  Thus,  we  label  a  state  wuii  (J  it f  theio  is  a  path  from  dial  state  to  some 
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node  of  a  fair  strongly  connected  component.  As  usual  we  design  the  algorithm  so  that  after  it 
terminates  each  state  will  he  labeled  with  the  subformulas  of  f0  true  in  that  state.  We  consider  the  two 
interesting  cases  where  /  is  a  subformula  of  /  and  cither  /  =  L[/  U f  ]  or  /  =  F.G/.  We  assume 
that  the  states  have  already  been  labeled  with  the  immediate  subformulas  of  /by  an  earlier  stage  of  the 
algorithm. 

1.  /  =  Y[f.  U/| :  /  is  true  in  a  state  iff  the  CT1.  formula  F[/  U  (J\  A  <21  ]  is  true  in  that  state, 
and  tins  can  be  determined  using  the  CTl.  model  checker.  Again,  state  s  is  labeled  with  /  iff 
/is  true  in  that  suite. 

2.  /•-  F.G( /);  lo  determine  if  jN=  KG ( / )  we  use  tire  procedure  described  in  section  3  to 
check  st=  F.G(  /A  Q)  in  the  structure  with  the  additional  proposition  Q. 

It  is  easy  to  see  that  the  above  algorithm  runs  in  time  O{length(f0)-  ( 1 5'  |  +  |  R  | )) . 

Theorem  4:  There  is  an  algorithm  for  determining  whether  a  CTL^  formula  fQ  is  true  in  state  s  of  the 
structure  M  =  (S,  R,  /,,  F)  with  F  as  the  set  of  fairness  constraints  that  runs  in  time 
O(!engtti(f0)(\S\  +|/?|)). 

5.  An  Example 

In  this  section  wc  illustrate  how  the  model  checker  can  be  used  to  verify  a  simple,  but  not  entirely 
trivial,  concurrent  program.  The  example  is  a  two  process  mutual  exclusion  program  that  was  manually 
proved  correct  using  linear  temporal  logic  by  Owicki  and  Lamport  in  [Owicki  &  Lamport  82].  The 
program,  expressed  in  a  variant  of  the  CSP  programming  language  [Hoarc  73],  is  shown  in  Figure  5-1. 
In  this  version  of  CSP  processes  may  have  global  variables  (e.g.  pi  and  p2),  and  assignments  to  such 
variables  arc  assumed  to  be  atomic.  Since  our  verification  technique  can  only  be  used  to  analyze  finite 
state  concurrent  systems,  we  require  that  all  variables  be  boolean  and  that  all  messages  between 
processes  be  signals.  Labels  (e.g.  NCI  and  NC2)  are  used  to  indicate  that  flow  of  control  has  readied  a 
particular  point  in  some  process.  In  our  example  there  are  two  processes  .57  and  S2.  and  each  process 
has  three  code  regions:  a  noncriiical  region  NCi  in  which  the  process  computes  some  data  values  that  it 
wishes  to  share  with  the  other  process,  a  trying  region  Ti  in  which  the  process  executes  a  protocol  to 
obtain  entry  into  the  critical  section,  and  a  critical  section  CSi  in  which  the  process  updates  shared 
variables.  To  prevent  a  race  condition  that  might  result  in  unpredictable  values  being  assigned  to  the 
shared  variables,  only  one  process  is  allowed  to  be  in  its  critical  section  at  any  given  time.  Note  that  the 
two  processes  arc  different;  hence  this  is  not  a  symmetric  solution  to  the  mutual  exclusion  problem. 
When  the  CSP  program  is  compiled  a  state  graph  with  77  suite-,  is  obtained.  Mthoueh  this  is  to *t  an 
extremely  large  state  machine,  it  would  ncvcilhclcss  be  quite  tedious  toi  a  human  to  debug. 


10 


We  initially  run  the  verifier  without  any  fairness  constraints--See  Figure  5-2.  Wc  first  check  to  sec  if 
both  processes  arc  ever  in  their  critical  regions  at  the  same  time.  This  property  is  succintly  expressed  by 
the  CTI.  formula  FF(CS1  A  CS2).  The  verifier  rapidly  determines  that  the  formula  is  falsc-hcncc.  the 
program  does  guarcntcc  mutual  exclusion.  Time  is  measured  in  1/60  of  a  second.  The  first  component 
measures  userepu  time.  The  second  component  measures  system  epu  time.  Wc  next  check  for  absence 
of  deadlock.  This  is  expressed  by  the  formula  AG(FF(C.S1  v  CS2)).  The  verifier  determines  that  this 
formula  is  satisfied:  thus,  from  any  state  that  is  reachable  from  the  initial  state  it  is  always  possible  to  get 
to  cither  CSV  or  CS2. 

Absence  of  starvation  for  process  1  is  expressed  by  the  formula  AG  (71  —  AF  CAT) .  T  his  property  is 
not  satisfied  without  a  fairness  constraint.  The  reason  is  quite  simple.  When  wc  build  the  global  state 
graph  for  the  program  wc  do  not  make  any  assumptions  about  the  relative  speeds  of  the  two  processes. 
T  hus,  die  second  process  can  make  any  number  of  steps  between  steps  of  the  first  process.  In  fact,  the 
second  process  can  even  run  forever,  thereby  preventing  die  first  process  from  ever  making  anodicr 
step.  We  can  rule  out  the  second  type  of  behavior  by  means  of  fairness  constraints  which  require  diat 
each  process  be  given  a  chance  to  execute  infinitely  often.  In  Figure  5-3  we  restart  the  verifier  with 
several  fairness  constraints  that  prevent  cither  process  from  remaining  forever  at  die  same  statement 
while  enabled  to  make  a  step.  Under  these  assumptions  the  first  process  will  never  starve.  I  lowevcr.  the 
possibility  of  starvation  still  exists  for  the  second  process. 

A  good  solution  to  the  mutual  exclusion  problem  should  not  require  that  processes  alternate  entry 

into  dicir  critical  regions:  CAT,  CS1 ,  CAT,  CA’2 . In  order  to  test  diat  the  algorithm  given  in  Figure 

5-1  does  not  require  strict  alternation,  wc  check  die  formula 
AG  (CAT  -»  AICS1U(^CS1AA[-CATUCA2))]). 

This  formula  asserts  that  if  process  1  enters  its  critical  section  and  subsequently  leaves  it,  then  it  cannot 
enter  it  again  until  process  2  has  entered  its  critical  section.  The  verifier  determines  that  die  formula  is 
false  in  less  than  a  second.  This  example  shows  how  die  basic  temporal  operators,  particularly  the  until 
operator,  can  be  nested  to  express  complicated  timing  properties. 

Finally,  the  verifier  has  a  counterexample  feature  (that  is  not  shown  in  the  transcripts).  When  this 
feature  is  enabled  and  the  model  checker  determines  that  a  formula  is  false,  it  will  attempt  to  find  a  p.tdi 
in  the  state  graph  which  demonstrates  that  the  negation  of  die  formula  is  true.  For  example,  if  the 
formula  has  die  form  AG(/),  our  system  will  produce  a  path  to  a  suite  in  which  -/holds.  For 
instance,  when  die  verifier  determines  that  die  last  formula  above  is  false,  it  prints  out  an  execution  of 


p 1 . p 2  :  bool  ; 

NCl,NC2,TllT2.T2a,CSl,CS2: 

[ 

SI ,  S  2  :  process; 

SI  ::  [ 

pi  : =  false; 

*[ 

true  -> 
<<NC  1>> 

«T1» 
<<CS  1>> 


] 


] 


1  abel ; 


skip;  --noncr i t i cal  section  1 
pi  :=  true; 

*[  p2  ->  skip]; 

skip;  --critical  section  1 

pi  :=  false 


S2  :  :  [ 

p2  :=  false; 

*c 

true  -> 

<<NC2>>  skip;  - -noncr i t i cal  section2 
p2  :=  true; 

<<TZ>>  *[  pi  -> 

p2  : -  false; 

<<T2a>>  * [ p  1  ->  skip  ]; 

p2  :-  true 

]; 

<<CS2>>  skip;  --critical  section  2 
p2  :=  false 

] 

3 


Figure  5-1:  Two  process  mutual  exclusion  program. 


CTL  MODEL  CHECKER  (C  version  2.5) 


|=  EF(CSl  &  CS2). 

The  equation  is  FALSE. 

time:  (2  4) 

|=  AG(EF(CSl  |  CS2)). 

I  he  equation  is  1  RUE. 

lime:  (4  2) 

AG(T1  ->  AFCSl). 

I  he  equation  is  FALSE. 

time:  (17  12) 

Figure  5-2:  Transcript  of  model  checker  execution  ( without  fairness 

constraint). 


Fairness  constraint:  ~NC1. 

Fairness  constraint:  ~NC2. 

Fairness  constraint:  ~CS1. 

Fairness  constraint:  -CS2. 

Fairness  constraint:  -  IT  |  P7. 

Fairness  constraint:  ~'12 1  pi. 

Fairness  constraint:  ~T2  |  ~pl  |  T2a. 

Fairness  constraint: . 

|=  AG(T1  •>  AFCSl). 

The  equation  is  TRUE. 

time:  (10  0) 

|=  AG(T2->  AFCS2). 

The  equation  is  FALSE. 

time:  (29  9) 

|=  AG(  CS1  •>  A[CS1  U  (~CS1  &  A(~CSi  U  C$2])j). 
The  equation  is  FALSE, 
time:  (38  17) 


Figure  5-3:  Transcript  of  model  checker  execution  (with  fairness  constraint). 
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the  mutual  exclusion  program  in  which  process  1  enters  its  critical  region,  leaves,  and  reenters  without 
process  2  entering  its  critical  section  in  the  meantime.  This  feature  is  quite  useful  for  debugging 

purposes. 

6.  Other  Approaches 

Several  papers  have  considered  the  model  checking  problem  for  linear  temporal  logic  fornmljs.  l  et 
M  =  (.S'.  R,  I.)  be  a  Kripkc  Structure  with  s0  e  S’,  and  let  A/ be  a  linear  temporal  logic  foimui.i.  1  bus. 
/  is  a  restricted  path  formula  in  which  die  only  state  subfonnulas  are  atomic  propositions.  We  wish  to 
determine  if  A/,^  1=  A/.  Notice  that  M.s  t=  A/ iff  A/.s  M=  -■F ->/.  Consequent'.),  it  is  Miiiiucnt  to  be 
able  to  check  die  truth  of  formulas  of  the  form  K/ where /is  a  restricted  path  formula.  In  general,  dus 
problem  is  PSPACK-compIctc  [Sistla  &  Clarke  86],  Although  the  proof  of  this  PSPACI  -completeness 
result  is  beyond  the  scope  of  our  survey,  it  is  easy  to  see  that  the  model  checking  problem  is  NP-hard 
for  formulas  of  the  form  F./  where  /is  restricted  path  formula.  We  show  diat  the  directed  Hamiltonian 
path  problem  is  reducible  to  the  problem  of  determining  whether  ,U,jN  /where 

•  M  is  a  finite  structure, 

•  s  is  a  state  in  M  and 

•  /is  the  asserdon  (using  atomic  propositions  px . /?„): 

V[¥p\  A  ...  A  Vpn  A  G(/>,  -*  XG^p)  A  ...  A  C (pn  -*  XG-’/k)!- 

Consider  an  arbitrary  directed  graph  G  =  ( V,  ,()  where  V  =  (v. . v,,} .  We  obtain  a  staicturc 

from  G  by  making  proposition  p,  hold  at  node  v,  and  false  at  all  other  nodes  (for  1  <i<n),  and  by 
adding  a  source  node  ul  from  which  all  v(  arc  accessible  (but  not  vice  versa)  and  a  sink  node  which 
is  accessible  from  all  v,  (but  not  vice  versa).  Toimallv,  let  the  structure  A f  =  (U.  R.  I.)  consist  of 

V  =  f'U{u,.i/2i  where  Wj.UjCK; 

0  =  /1U  {(n,,  Vj)|  vy  €  U  {( v(,  w.)| vt  €  nij{(u,  m)};and 

L  is  an  assignment  of  propositions  to  states  such  that 

•  p,  is  true  in  v,  for  1  <  i<  n 

•  pj  is  false  in  vt  for  1  <  ij<  n,  t+j 

•  />,  is  false  in  u  .u,  for  1  <  i£  n 


*.s  aVa  aV/ 
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It  is  easy  to  sec  that  A f.ul  N  / iff  there  is  a  directed  infinite  path  in  M  starting  at  u,  which  goes  through 
all  v,  e  V  exaedy  once  and  ends  in  die  self  loop  diough  ul.  Note  that  the  formula  /  in  die  above 
construction  has  essentially  the  same  si/c  as  die  as  die  graph  (7.  Suppose  dial  the  length  of  the  formula 
to  he  checked  was  known  to  be  much  smaller  than  the  si/c  of  the  Kripke  structure  under  consideration. 
Would  the  complexity  still  be  high  in  this  case?  A  careful  analysis  by  Lichtenstein  and  Pnueli 
[Lichtenstein  &  Pnucli  85)  showed  diat  aldiough  the  complexity  is  apparently  exponential  in  the  length 
of  the  formula,  it  is  linear  in  the  si/c  of  the  global  state  graph.  We  briefly  describe  their  results  below. 

Let  /be  a  restricted  path  formula,  '['lie  chuuv  off  Cl  (/).  is  die  smallest  set  of  formulas  containing  f 
and  satisfying: 

•  -/  €  Cl.(f)  iff  /  €  Cl  (J') 

.  if  /  v/2 1  r/ (/) .  then  /. /  €  Cl (/) 

•  if  X/  e  C/.(/),  then  /  e  Cl.(f) 

•  if  -X/  €  C/.(/).  dicn  X-/  €  Cl(f) 

4  if/U /  c  CL(f),  dien  /./.  X[/U/J  €  CI(J) 

It  can  be  shown  that  the  si/e  of  Cl(f)  is  5  lcngih( /). 

An  atom  is  a  pair  A  =  h A)  with  sAiS  and  /•  <CC  L(/)U AP  such  that: 

•  for  each  proposition  Q  €  Al \  Q  €  lA  iff  Q  ?  /( sA ) 

•  for  every  /  €  CZ.(/),  /  €  FA  iff  -■/  <  F^ 

•  for  every  /,/  €  Cl.(f).  /v/  e  iff  /  or  /  €  FA 

•  for  every  -’X/  €  CL(f),  ->X/  £  F  A  iff  X->/  £  F< 

•  for  every  /./  €  C/-(/),/L'/  £  FH  iff  /  €  F4  or  /.  X[/L/]  £  /  ., 

Now,  a  graph  G  is  constructed  with  the  set  of  atoms  as  the  set  of \crticcs.  (A  H)  is  an  edge  of  (/  if t 
(sA.Sff)eR  and  for  every  formula /,  if  X/cF<t  ilicn  /  t  Fn.  An  eventual:/}  st^ucncc  is  an  infinite 
path  it  in  (i  such  that  if  /  U  /  £  F^  for  some  atom  .!  on  ir ,  dien  diere  cxnis  an  atom  /?,  readable 
from  I  along  it ,  Midi  that  /  £  Fj. 

Lemma  5:  A/,\  h—  K/  iff  there  exists  an  eventuality  sequence  Marling  at  an  atom  ( c  F)  such  that 
fV  F. 


ajgaaia^^ 
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A  non-trivial  strongly  connected  component  C  of  the  graph  G  is  said  to  be  self-fulfilling  iff  for  every 
atom  A  in  C  and  for  every  f\lf  e  FA  dierc  exists  an  atom  II  in  C  such  that  f  e  Fg. 

Lemma  6:  M  s  N  V.f  iff  dierc  exists  an  atom  A  =  { s.  I')  in  (i  such  that  fc  I  and  there  exists  a  padi 
in  G  from  .1  to  a  self-fulfilling  strongly  connected  component. 

I.emma  6  he  used  as  the  basts  for  a  linear  temporal  logic  model  checking  algorithm.  I  his  algorithm 
has  die  time  complexity  M((|.S'|  F|W|)  2  1  icluensiein  and  I’nucli  furtiier  shewed  how  tins 

basic  algorithm  could  be  extended  to  handle  a  number  of  different  notions  of  fairness  with  essentially 
the  same  complexity. 

An  alternative  approach  due  to  Vardi  and  Wolpcr  (Vardi  &.  Wolper  86]  exploits  the  close  iclationship 
between  linear  temporal  logic  formulas  and  Buchi  automata.  A  Ihiclti  autenata  is  a  tuple 
A  =  (2,  S,  p.  S0,  F),  where 

•  2  is  an  alphabet. 

•  S  is  a  set  of  states. 

•  p  :  .S’x2  — •  2s  is  a  nomlcterministic  transition  function. 

•  S.t C.V  is  a  set  of  initial  states. 

•  /  C5  is  a  set  of  designated  states. 

A  run  of  A  on  an  infinite  word  w  =  a{a: . . .  is  a  sequence  . . .  where  i0r  and  s,  €  p<  s,_  .  </.) ,  for 
all  />  1 .  A  run  Vi  ■  •  ■  's  accepting  if  there  is  some  designated  state  that  repeats  infinitely  often,  i.e.,  for 
some  j  €  F  there  arc  infinitely  many  / 's  such  that  j(  =  s.  1  he  infinite  word  >v  is  accepted  by  .1  if  there  is 
an  acccpung  run  of  A  over  w.  The  set  of  infinite  words  accepted  by  .1  is  denoted  L(A).  The  following 
theorem  is  proved  in  [Vardi  A  Wolpcr  86], 

i.emma  7:  For  every  linear  temporal  formula  V /.  a  lluclii  automata  A,  can  be  constructed,  where 
2  =  2 ^  and  |S|  t  such  that  I(.l;)  is  exactly  die  set  of  computations  satisfying  the  formula 

f 

A  Kripkc  Structure  \t-{S,R,l.)  with  initial  state  t  .Scan  be  viewed  as  a  llticlii  automaton 
A  if  =  (2,  S.  {*„}.  p  .V)  where  2  --  2AP  and  s'  €  p(s.  a)  iff  (v,  s')  €  R  and  a  =  /.( s> .  Note  that  any 
infinite  run  of  this  automaton  is  accepting.  JL(  <u)  is  the  set  of  computations  of  Au.  Ihus.  m  order  to 
determine  whether  \fst=  \/  it  is  sufficient  to  check  whether  1(1  u)fS  ft  1^,)  is  empty  lliisc, mho 


determined  by  an  automata  theoretic  construction  with  essentially  the  same  time  complexity  as  the 
Pnueli  Lichtenstein  algorithm. 

One  of  the  expected  advantages  of  using  linear  temporal  logic  is  that  fairness  constraints  can  be 
handled  directly.  However,  if  fairness  constraints  arc  included  as  part  of  the  specifications,  the  formulas 
that  must  be  checked  will  in  general  be  quite  large.  For  instance,  consider  a  fairness  constraint  which 
requires  that  progress  he  made  from  any  state  in  die  program.  The  formula  that  expresses  this  property 
is 

\[  A  ~<G(at  s)  ->  <re  s/  of  specification)]. 

S€  -S 

which  has  si/c  (;(|.V|).  1'his  problem  was  realized  by  Lichtenstein  and  Pnueli  and  by  Vardi  and  Wolper. 
They  in  fact  handle  fairness  by  means  of  fairness  constraints  in  a  manner  very  similar  to  the  wav  it  is 
handled  in  [Clarke  ct  al  86a].  Another  problem  with  using  linear  temporal  logic  is  that  in  general  it  is 
impossible  to  handle  specifications  which  involve  existential  path  quantifiers.  Although  it  is  possible  to 
check  simple  formulas  of  the  form  F. /  where  f  is  a  restricted  paih  formula,  it  is  not  possible  to  check 
formulas  like  AG(F.F/),  which  is  used  to  express  absence  of  deadlock  in  die  example  in  section  5. 
Moreover,  model  checking  for  the  full  logic  CTL*  is  no  more  difficult  dian  for  linear  temporal  logic  as 
was  shown  py  Hmerson  and  Lei  [Fmcison  &  Lei  85]. 

Theorem  8:  If  we  arc  given  an  algorithm  AI.[TL  to  solve  the  model  checking  problem  for  linear 
temporal  logic,  then  we  can  construct- an  algorithm  A Lqtl  f°r  lu"  '°S‘C  ^’1  L  that  has  the  same 
order  of  complexity  as  AL[j £. 

7.  Applications 

Sequential  circuit  verification  is  a  natural  application  for  die  type  of  verifier  discussed  in  this  paper. 
Bochmann  (Bochmann  82]  was  probably  the  first  to  realize  the  usefulness  of  temporal  logic  for 
describing  the  behavior  of  circuits.  He  verified  an  implementation  of  a  seif-timed  arbiter  using  linear 
temporal  logic  and  what  he  called  "reachability  analysis.”  The  work  of  Malachi  and  Owicki  [Malaehi  & 
Owicki  81]  identified  additional  temporal  operators  required  to  express  interesting  properties  of  circuits 
and  also  gave  specifications  for  a  large  class  of  modules  used  in  self-timed  circuits.  Although  (hese 
researchers  contributed  significantly  toward  developing  an  adequate  notation  for  expressing  die 
correctness  of  sequential  circuits,  the  problem  of  mechanically  verifying  a  circuit  remained  unsolved. 

In  [Mishra  &.  Claike  85]  Clarke  and  Mishra  showed  how  die  HMC  algorithm  could  be  used  to  u'lity 
various  temporal  properties  of  asynchronous  circuits.  I  hey  developed  a  technique  for  extracting  a  Mate 
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graph  directly  from  a  wire-list  description  of  tire  circuit  (i.c.,  from  a  description  of  the  circuit  in  terms  of 
its  components  and  their  interconnections).  Tire  model  checker  was  then  used  to  show  that  state  graph 
satisfied  various  specifications  expressed  in  temporal  logic.  In  this  way  they  were  able  to  determine  drat 
a  self-timed  queue  element  described  in  Seitz'  chapter  of  Mead  and  Conway  [Seitz.  80]  did  not  satisfy  its 
specifications.  Their  work  was  later  extended  by  Browne.  Claike.  Dill,  and  Mishra  [Browne  ct  al 
86]  who  showed,  in  general,  how  a  mixed  gate  and  switch  level  circuit  simulator  could  be  used  to  extract 
a  state  graph  from  a  structural  description  of  a  sequential  circuit.  The  basic  simulation  algorithm  is 
shown  in  f  igure  7-1.  Circuits  are  usually  designed  under  the  assumption  that  certain  input  sequences 
and  combinations  will  not  occur.  I  heir  program  exploits  tins  observation  to  prevent  a  combinatorial 
explosion  in  die  number  of  states  that  are  generated,  by  allowing  the  user  to  specify  a  set  of  conditions 
under  which  die  inputs  can  change. 

{The  procedure  below  uses  a  hash  table  that  maps  node 
value  assignments  to  states. To  construct  the  state  machine, 
call  this  procedure  on  a  node_va  1  ue_as  s  i  gninen  t  for  the 
initial  state.} 

procedure  Bu  i  1  dGr aph (  Node_val  ue_as s  i  ynmen  t )  return  a  state 
begin 

if  there  is  a  state  for  the  node_val ue_as s i gnmen t 

already  in  the  table  then  return  the  state; 

else 

Create  a  new  state; 

Label  state  with  nodes  that  have  1  values; 

Store  state  and  node  values  together  in  hash  table; 

for  each  possible  input  assignment  do 

Combine  current  values  for  internal  nodes  and  input 
assignment  into  a  new  node_v,i  1  ue_as  s  i  gnmen  t ; 

Simulate  one  step  to  find  a  new  node  assignment; 

Call  BuildGraph  recursively  on  new  node  assignment; 

Add  value  returned  by  previous  line  to  successors  of 
current  state; 

end 

end 

end 

Figure  7*1:  Algorithm  For  Constructing  Kripke  Structure  I-'roin  Circuit 


The  circuit  simulator  in  (Browne  ct  ai  86]  used  a  unit-delay  timing  model  in  which  the  switching 
delays  of  all  the  transistors  and  gates  arc  assumed  to  be  equal.  While  a  unit-delay  model  is  satisfactory 
for  synchronous  circuits,  it  may  not  be  appropriate  for  asyncliionous  circuits.  In  [Dill  &  Clarke  S6]  I  )ill 
and  Clai  ke  showed  how  Kripke  structures  could  be  extracted  from  a  gate  level  description  of  a  circuit 
under  a  model  of  circuit  behavior  that  permitted  arbitrary  non-/cro  delays  to  he  associated  with  the 
outputs  of  the  gates.  Flic  basic  idea  behind  their  approach  is  quite  simple.  Con-adci  an  \\T)  a,. to  with 


two  inputs,  x  and  y ,  and  a  single  output  z.  Assume  that  the  gate  is  in  an  unstable  configuration  with  x 
low,  y  high,  and  z  high.  The  Kripke  structure  for  the  circuit  containing  this  gate  will  have  a  state 
corresponding  to  the  unstable  configuration  as  shown  in  Figure  7-2.  The  state  will  hare  a  self-loop  and 
a  transition  to  another  state  representing  a  stable  configuration  in  which  the  output  is  low.  Fairness 
constraints,  as  described  in  Section  4,  arc  used  to  insure  that  the  system  doesn't  remain  m  an  unstable 
configuration  forever.  In  tire  case  of  the  AND  gate,  it  is  sufficient  to  require  that  infinitely  often 
z  -  xAy. 


Figure  7-2:  Krikpc  structure  for  unstable  configuration  of  AND  gate. 

In  practice,  tire  arbitrary  delay  model  is  much  too  conservative.  Many  circuits  arc  \ilmcst  speed 
independent":  ['hey  do  not  appear  to  be  correct  under  a  pure  arbitrary  delay  model,  but  would  work 
given  reasonable  assumptions  about  the  relationships  between  the  delays.  When  the  circuit  designer 
has  a  great  deal  of  control  over  the  magnitudes  of  circuit  delays,  exploiting  more  detailed  knowledge  of 
circuit  timing  can  result  in  smaller  and  faster  circuits.  In  fact,  actual  circuits  often  rely  on  such 
assumptions.  In  [Urowne  et  al  85]  and  [Dill  86]  a  method  is  described  for  adding  such  assumptions  to  a 
circuit  description  and  incorporating  them  into  the  state-graph  construction.  Possible  timing  constraints 
include  constant  upper  and  lower  bounds  on  individual  delays,  and  bounds  on  tlte  differences  between 
delays.  Using  constraints  of  this  form,  one  can  say  for  example:  "the  delay  of  the  first  AM)  gate  is 
between  5  and  10  nanoseconds"  or  "the  delay  of  the  first  AND  gate  is  greater  titan  the  delay  of  tlte 
second  AND  gate."  The  state  graph  constructed  with  respect  to  a  particular  set  of  delay  assumptions 
rules  out  some  circuit  executions  which  would  be  allowed  under  an  arbitrary  delay  model.  Hence, 
formulas  in  CTL  which  might  not  have  been  true  in  an  arbitrary  delay  model  may  be  tine  with  respect 
to  particular  delay  assumptions  (because  all  the  counterexample  paths  are  ruled  out  by  the  deity 
assumptions).  This  technique  was  applied  to  a  patented  asynchronous  queue  cell  in  [Browne  et  a  I  S  5). 
The  authors  determined  that  (lie  circuit  did  not  meet  its  specifications  tinder  the  arbitrary  element  delay 


mode!.  However,  under  die  assumption  that  the  input  was  slower  than  two  of  the  circuit  gates,  they 
showed  that  the  circuit  met  its  temporal  logic  specifications. 

An  alternative  approach  obtains  the  state  diagram  by  compilation  from  a  specification  of  the  original 
(synchronous)  circuit  in  a  simple  programming  language-like  notation.  Browne  and  Clarke  (  [Browne  et 
al  86],  [Browne  &.  Clarke  86])  use  a  Pascal-like  state  machine  description  language  called  SV1L  for  this 
purpose,  (he  language  includes  the  standard  control  structures  if,  while,  and  loop/cxit.  A  cobcgin 
statement  is  also  provided  for  simultaneous  execution  of  statements  in  lock-step.  8m.ee  SMI.  progiams 
will  ultimately  be  implemented  in  hardware,  the  only  data  types  permitted  arc  bonlcatt  and  (bounded) 
integer.  The  output  of  die  S.ML  compiler  is  a  deterministic  Moore  Machine  that  can  be  automatically 
implemented  as  a  PI  A.  PAI..  or  a  ROM.  The  output  can  also  be  analyzed  for  correctness  using  the 
hMC  algorithm.  In  [Browne  86]  Browne  describes  a  specialized  version  of  die  PMC  algorithm  that  can 
check  Moore  machines  much  more  rapidly  than  the  original  algorithm. 

Another  potential  area  of  application  is  the  verification  of  network  communication  protocols.  The 
alternating  bit  protocol  [Bartlet  et  al  69]  for  reliable  transmission  of  messages  by  a  noisy  communication 
channel  is  a  simple  example  of  such  an  algorithm.  By  using  the  CTL  model  checking  procedure  it  is 
possible  to  determine  in  a  few  seconds  whether  this  protocol  meets  its  specific, '(ions  [Clarke  et  al  SCi], 
Sifakis  at  Grenoble  [Quicllc  &  Sifakis  81]  and  Kurshan  at  Bell  l  abs  [Kurshan  86]  have  also  considered 
applications  involving  network  protocols.  The  delay  assumptions  mentioned  above  may  be  useful  for 
describing  die  real-lime  behavior  of  such  protocols. 

8.  Conclusion 

Although  the  verification  technique  described  in  this  paper  has  already  been  used  to  find  some 
nontrivial  errors  in  circuit  designs  and  communications  protocols,  more  research  needs  to  be  done 
before  it  will  become  a  truly  practical  debugging  tool  for  use  by  system  designer.  One  pioblem  is  die 
expressibility  of  the  underlying  temporal  logic.  For  circuit  specification  itm.ng  Jiagrams  may  be  more 
natural  to  use  than  temporal  logic  formulas.  Of  course,  temporal  logic  is  more  general  since  there  is  no 
analogue  of  negation,  disjunction,  or  conjunction  for  timing  diagrams.  It  may  be  possible  to  either 
systematically  translate  timing  diagrams  into  temporal  logic  formulas  or  check  them  directly  using  an 
algorithm  similar  to  the  one  used  by  the  model  checker.  If  so,  dus  would  simplify  the  task  of  specifying 
a  complicated  circuit  and  also  allow  the  designer  to  be  moie  confident  that  specifications  actually  mean 
what  he  dunks  they  mean. 


The  most  important  problem,  however,  is  the  slate  explosion  problem.  There  arc  several  different 
strategies  for  handling  this  problem.  In  verifying  as>nchronous  circuits,  for  example,  buggy  circuits 
sometimes  result  in  much  larger  state  graphs  than  correct  circuits.  This  happens  because  the  activity  in 
tire  circuit  is  much  more  disordered  after  an  error  has  occuired.  One  possible  solution  m  this  case  is  to 
run  the  program  which  builds  the  suite  graph  and  the  model  checker  as  co-routines,  creating  states  only 
as  they  need  to  be  referenced  by  the  model-checker.  In  [Dill  8b]  this  technique  is  called  lazy  stale 
generation,  by  analogy  to  lazy  evaluation  in  programming  language  implementations.  By  using  this 
method,  an  error  could  be  discovered  and  reported  after  constructing  only  a  small  part  of  the  enure 
state  graph:  tins  would  not  only  speed  up  the  verification  process,  it  would  also  make  it  pos-JMe  to 
verify  some  circuits  which  could  not  be  verified  if  the  entire  giapii  had  to  be  constructed. 

Another  approach  to  the  suite  explosion  problem  is  to  exploit  the  hierarchical  structure  of  complex 
finite  state  concurrent  systems.  If  an  appropriate  subset  of  Cl  I.  is  used  t  [Mishra  &  Clarke  85],  [Clarke 
ct  al  86bj),  dien  lower  level  subcircuius  can  be  simplified  by  "hiding"  some  of  their  internal  nodes  (more 
precisely,  making  it  illegal  to  use  diem  in  temporal  logic  formulas)  and  merging  groups  of  suites  that 
become  indistinguishable  into  single  state.  Preliminary  research  in  [Mishra  it  Clarke  85]  indicates  that 
by  using  this  technique  it  may  be  possible  to  cut-down  dramatically  on  the  number  of  states  dial  need 
to  be  examined. 


Finally,  special  techniques  may  be  appropriate  for  concurrent  systems  that  arc  composed  of  many 
identical  processes.  Consider,  for  example,  a  distributed  mutual  exclusion  algorithm  for  piucc'-scs 
arranged  in  a  ring  network  in  which  mutual  exclusion  is  guaranteed  by  means  of  a  token  thar  is  passed 
around  the  ring  ( [Dijkstra  85[.  [Kurshan  85),  [Martin  85]).  A  strategy  that  is  often  used  for  debugging 
such  systems  is  to  consider  first  a  reduced  system  with  one  or  two  processes.  If  it  is  possible  u  ■  show  that 
the  reduced  system  is  correct  and  if  the  individual  processes  arc  really  identical,  then  one  is  tempted  to 
conclude  that  die  enure  system  will  be  coneU.  InJClaikc  et  al  Gob]  an  attempt  is  made  to  pro.  ide  a 

solid  theoretical  basis  that  will  prevent  fallacious  conclusions  in  arguments  of  tins  type.  I  lie  authors 

•  • 

describe  a  temporal  logic  called  Indexed  CTI  ,  or  1CTI.  for  specifying  networks  of  identical  processes. 
1  lie  logic  includes  all  of  CTI.  with  die  exception  of  the  nexttime  operator;  in  addition,  it  permits 
formulas  of  the  form  A  J\i)  and  V y ( r )  where  f(i)  is  a  formula  in  which  all  of  the  atomic  pr< 'positions 

/  l 

arc  subsenpted  by  /.  A  Knpkc  structure  for  a  family  of  .V  identical  processes  may  be  obtained  as  a 
product  of  the  suite  graphs  of  the  individual  processes.  Instances  of  the  same  atomic  p:<> position  in 
ditfercnt  processes  are  distinguished  by  using  the  number  of  the  process  as  a  subscnpt.  thus.  I, 
represents  the  instance  of  atomic  pioposition  I  associated  with  process  5. 


Since  a  closed  formula  of  the  new  logic  cannot  contain  any  atomic  propositions  with  constant  index 
values,  it  is  impossible  to  refer  to  a  specific  process  by  writing  such  a  formula.  Hence,  changing  the 
number  of  processes  in  a  family  of  identical  processes  should  not  effect  tire  truth  of  a  formula  in  the 
logic.  This  intuitive  idea  is  made  precise  by  introducing  a  new  notion  of  bisimulation  [Milner 
79]  between  two  Kripke  structures  with  the  same  set  of  indexed  propositions  but  diffcicnt  sets  of  index 
values.  It  is  possible  to  prove  that  if  two  structures  correspond  in  this  manner,  a  closed  formula  of 
Indexed  CTL*  will  be  true  in  the  initial  state  of  one  if  and  only  if  it  is  true  in  the  initial  state  of  die 
other. 

These  ideas  arc  illustrated  in  [Clarke  ct  al  86b]  by  considering  die  distributed  mutual  exclusion 
algorithm  mentioned  above.  The  atomic  proposition  c,  is  true  when  the  /- th  process  is  in  its  critical 
region,  and  the  atomic  proposition  dt  is  true  when  the  /-th  process  is  delayed  waiting  to  enter  its  critical 
region.  A  typical  requirement  for  such  a  system  is  that  a  process  waiting  to  enter  its  critical  region  will 

eventually  do  so.  This  condition  is  easily  expressed  in  ICTL*  by  the  formula  A.\G(t/,=>  AFc,).  The 

.  i 

results  of  [Clarke  ct  al  86b]  can  be  used  to  show  that  exactly  the  same  ICTL  formulas  hold  in  a  network 
with  1000  processes  as  hold  in  a  network  with  two  processes.  The  EMC  algorithm  can  be  used  to  check 
automatically  that  die  above  formula  holds  in  networks  of  size-two  and  conclude  diat  it  will  also  hold  in 
networks  of  size  1000.  At  present  this  methodology  has  only  been  partially  automated,  however.  The 
bisimulation  must  be  established  by  hand  and  this  generally  requires  some  representation  of  die  larger 
Kripke  structure.  Several  researchers  arc  attempting  to  find  a  way  of  automating  this  phase  in  a  manner 
that  avoids  building  the  larger  Kripke  structure. 

Other  techniques  for  avoiding  the  state  explosion  problem  arc  being  investigated  by  Kurshan  ar.d 
Wolpcr.  In  Kurshan's  system  [Kurshan  85]  diis  problem  is  handled  by  using  a  homomorphism  to 
collapse  a  large  state  machine  into  a  much  smaller  one  while  preserving  those  properties  diat  arc 
impodant  for  verification.  Since  Kurshan  docs  not  use  temporal  logic  formulas  for  specification,  lie  lias 
no  analogue  of  the  indexed  formulas  or  of  the  bisimulation  dieorem  used  in  [Clarke  ct  al  86b].  Wolpcr 
[Wolpcr  86]  considers  a  logic  somewhat  like  ICTL*  for  reasoning  about  programs  that  arc  d.ua- 
independent;  however,  his  indexed  variables  range  over  data  elements,  not  over  processes.  Also,  there 
is  no  notion  of  correspondence  between  structures  in  his  work.  Some  ultimate  limitations  on  this  tv  pc  of 
reasoning  arc  discussed  in  Apt  and  Kozen  [Apt  &  Kozcn  86]. 
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